Senior Data Platform Engineer - Remote Data Pipelines (AWS)

Komodo Health, Inc.; San Francisco (CA); Hybrid; USD 196,000 - 230,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

401(k) with match

Disability insurance

Life insurance

Flexible time off

Leaves of absence

Job summary

Komodo Health is seeking a Senior Data Platform Engineer to design, build, and operate large-scale data pipelines powering the Healthcare Map. You will work with Python, SQL, Spark, Airflow on AWS, and contribute to data quality, observability, and scalable data solutions.

You’ll join the Data Foundations team with a hybrid model (SF/NYC hub and US remote options), helping advance data products, governance, and AI-augmented engineering practices across the company.

Qualifications

• Strong hands-on experience building, owning, and debugging production data pipelines at multi-terabyte scale.
• Advanced Python and SQL for large-scale data processing.
• Deep production experience with Airflow, including operating and scaling it.
• Understanding of columnar vs row formats and open table formats like Iceberg, Delta Lake, or Hudi.
• Experience with Spark or a comparable distributed framework.
• Designing and operating data systems in AWS.
• Strong root-cause analysis and production troubleshooting.
• Ability to communicate technical trade-offs clearly with stakeholders.
• Comfort using AI-assisted engineering tools for coding and debugging.

Responsibilities

• Build, operate, and optimize large-scale batch pipelines using Python, SQL, Spark, and Airflow on AWS.
• Make storage and data layout decisions to keep datasets fast and cost-efficient.
• Ingest data from external sources and handle schema changes.
• Transform healthcare data into trusted serving-ready data products for customers and internal teams.
• Strengthen reliability with data quality checks, lineage, observability, and alerts.
• Debug complex data, system, and performance issues across workflows.
• Contribute to system design, testing, documentation, CI/CD, and production support.

Skills

Strong communication

Problem solving

Team collaboration

Mentorship

Adaptability

AI-assisted engineering

Tools

Python

SQL

Airflow

Spark

AWS

Iceberg/Delta Lake/Hudi

Job description

Komodo Health is seeking a Senior Data Platform Engineer to design, build, and operate large-scale data pipelines powering the Healthcare Map. You will work with Python, SQL, Spark, Airflow on AWS, and contribute to data quality, observability, and scalable data solutions.

You’ll join the Data Foundations team with a hybrid model (SF/NYC hub and US remote options), helping advance data products, governance, and AI-augmented engineering practices across the company.