Lead Data Engineer for Real-Time ML Pipelines

eHealth; United States; Remote; USD 115,000 - 144,000; Full time

Benefits offered by this job

Medical/Dental/Vision

401K with matching

Tuition reimbursement

Employee stock purchase program

12 company paid holidays

Flexible time off

Job summary

eHealth is seeking a Senior Data Engineer to lead the design of scalable data pipelines and ML-ready data solutions. You will collaborate with data scientists, software engineers, and business stakeholders to ensure reliable, accessible data powering critical decisions.

In a fast-paced, regulated healthcare environment, you will mentor engineers, drive observability, and advance best practices across the data team while leveraging AWS, Spark, Kafka, and dbt to deliver high-performance analytics

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
• 5+ years of data engineering experience; 3+ years with a Master’s degree.
• Expert SQL for complex queries and performance tuning on large datasets.
• Strong Python or Scala programming with CI/CD and testing best practices.
• Experience architecting cloud-native data platforms (Snowflake, BigQuery, Redshift, Databricks).
• Hands-on with dbt, Spark, Informatica, Matillion; NoSQL databases.
• Experience with AWS data services (S3, Glue, Lambda, Redshift, EMR).
• Communication skills to influence stakeholders and guide teams.

Responsibilities

• Serve as SME on data ecosystem and guide architectural decisions.
• Architect, build, and maintain scalable data pipelines and real-time streaming.
• Design workflow automation and orchestration with Airflow or Matillion.
• Lead design for production ML pipelines and model-serving APIs.
• Mentor engineers and establish coding standards and reviews.
• Drive data quality, observability, lineage, and governance.
• Collaborate with DevOps on platform performance and security.
• Evaluate new data tools and provide build-vs-buy recommendations.

Skills

SQL

Python/Scala

Cloud AWS

ETL/ELT

Data pipelines

Airflow/Matillion

API development

CI/CD

Education

Bachelor’s or Master’s in CS/Engineering

Tools

Spark

Kafka

dbt

Snowflake/BigQuery/Redshift/Databricks

GitHub Copilot

Job description

eHealth is seeking a Senior Data Engineer to lead the design of scalable data pipelines and ML-ready data solutions. You will collaborate with data scientists, software engineers, and business stakeholders to ensure reliable, accessible data powering critical decisions.

In a fast-paced, regulated healthcare environment, you will mentor engineers, drive observability, and advance best practices across the data team while leveraging AWS, Spark, Kafka, and dbt to deliver high-performance analytics