Lead Data Engineer - Spark & PySpark Platform Lead
HiLabs
14 days+
Get more replies from employers
Send a job-specific resume in minutes.
Benefits offered by this job
Job summary
A cutting-edge healthcare data platform company is seeking a Lead Data Engineer with expertise in Big Data technologies. This role involves leading the end-to-end design of data platforms, and building scalable systems using Spark and PySpark. Candidates should have 6 to 10 years of experience in large-scale architecture and distributed systems. The position is onsite in Bethesda, Maryland, offering competitive salary and comprehensive benefits, along with opportunities for mentorship and growth.
Qualifications
- 6 to 10 years of hands-on experience building and scaling Big Data applications.
- Strong expertise in Spark and PySpark.
- Experience with big data ecosystem tools like Hive and Solr.
- Solid understanding of large-scale architecture.
Responsibilities
- Lead end-to-end design and development of Big Data platforms.
- Architect and build high-performance distributed data systems.
- Collaborate closely with product, analytics, and DevOps teams.
- Mentor engineers and drive coding standards.
Skills
Spark
PySpark
Distributed systems
Data engineering
Workflow orchestration tools
Relational databases
AWS
Communication
Education
Bachelor’s or Master’s degree in Computer Science or related discipline
Tools
Apache Solr
Hive
Elasticsearch
Airflow
CI/CD tools
Job description
A cutting-edge healthcare data platform company is seeking a Lead Data Engineer with expertise in Big Data technologies. This role involves leading the end-to-end design of data platforms, and building scalable systems using Spark and PySpark. Candidates should have 6 to 10 years of experience in large-scale architecture and distributed systems. The position is onsite in Bethesda, Maryland, offering competitive salary and comprehensive benefits, along with opportunities for mentorship and growth.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
