Senior Data Platform Engineer – AI-Powered Pipelines

Carrum Health; United States; On-site; USD 180,000 - 250,000; Full time

Benefits offered by this job

Stock options

Performance bonus

Flexible schedules

Remote work

Office options in Chicago and San Frnc

Flexible PTO

Parental leave

Health, vision, and dental insurance

EAP

401K

Job summary

Carrum Health is seeking a senior platform engineer to own a Ruby/Rake‑based data pipeline evolving toward Python and Spark. You will build production services, tooling, and AI‑assisted ingestion across 230+ carrier integrations, while improving observability and deployment automation.

You’ll work in a remote‑first environment across the United States, collaborate with data scientists and engineers, and help modernize the platform with Databricks on the roadmap.

Qualifications

• 5+ years of professional software engineering experience.
• Strong Python and SQL; production‑grade practices.
• Experience with AWS stack (ECS, Batch, Lambda, S3, RDS, Athena).
• Terraform and dbt proficiency.
• Familiarity with observability and AI tooling in engineering workflows.

Responsibilities

• Own and extend the data pipeline platform with 230+ carrier integrations.
• Build production platform services (Python/SQL, Lambda, data services).
• Develop platform tooling for automation, testing, and data models.
• Design Terraform modules and deployment pipelines for reliability.
• Implement structured logging, monitoring, and data quality checks.
• Leverage Bedrock/Claude AI for ingestion, classification, and transformation tasks.
• Reduce operational toil by automating recurring data loads and provider switches.

Skills

Python

SQL

AWS

Terraform

dbt

Data pipelines

CI/CD

Debugging

AI tooling

Tools

Datadog

Spark

Eigen AWS tools

Job description

Carrum Health is seeking a senior platform engineer to own a Ruby/Rake‑based data pipeline evolving toward Python and Spark. You will build production services, tooling, and AI‑assisted ingestion across 230+ carrier integrations, while improving observability and deployment automation.

You’ll work in a remote‑first environment across the United States, collaborate with data scientists and engineers, and help modernize the platform with Databricks on the roadmap.