Senior Data Platform Engineer — AI-Driven Pipelines

Speedrun Talent Network; San Francisco (CA); Hybrid; USD 170,000 - 200,000; Full time

Job summary

Komodo Health is seeking a Senior Data Platform Engineer to design, build, and operate large-scale data pipelines powering the Healthcare Map. You’ll transform massive healthcare datasets into reliable data assets and contribute to core data systems that enable AI/ML use cases.

Open to US remote or SF/NYC hybrid, with base pay in the US ranging from $170,000 to $200,000 annually (and higher for SF/NYC). Komodo emphasizes AI‑augmented engineering and strong data governance, with a broad benefits

Qualifications

• Extensive experience building production data pipelines at multi-terabyte scale.
• Proficient in Python, SQL, and large-scale data processing.
• Deep expertise with Airflow and AWS data services.
• Strong troubleshooting and data correctness mindset.

Responsibilities

• Build, operate, and optimize large-scale batch pipelines with Python, SQL, Spark, Airflow on AWS.
• Decide storage formats, partitioning, and sizing for fast queries and cost efficiency.
• Ingest data from multiple sources and manage schema changes.
• Transform claims, EHR, and reference data into serving-ready datasets.
• Enhance reliability with data quality checks, lineage, observability, and alerts.
• Debug complex data and performance issues in heavy workloads.
• Contribute to design, testing, documentation, CI/CD, and production support.

Skills

Python

SQL

Spark

Airflow

AWS

Data pipelines

Data quality

Troubleshooting

Communication

AI tools

Tools

Iceberg/Delta Lake/Hudi

Docker

Kubernetes

CI/CD

Job description

Komodo Health is seeking a Senior Data Platform Engineer to design, build, and operate large-scale data pipelines powering the Healthcare Map. You’ll transform massive healthcare datasets into reliable data assets and contribute to core data systems that enable AI/ML use cases.

Open to US remote or SF/NYC hybrid, with base pay in the US ranging from $170,000 to $200,000 annually (and higher for SF/NYC). Komodo emphasizes AI‑augmented engineering and strong data governance, with a broad benefits