Senior Data Platform Engineer (Databricks, Python)

CVS Health; Atlanta (GA); In loco; USD 102.000 - 204.000; Tempo pieno

Descrizione del lavoro

CVS Health is looking for a Senior Data Platform Engineer to build, run, and evolve our Azure Databricks data platform that ingests, validates, and protects data for the business domain data engineering teams.

You will partner with domain teams to onboard data sources, develop shared Python libraries, and own CI/CD pipelines (GitHub Actions) and Terraform for Azure/Databricks infrastructure, while enforcing security through Unity Catalog and Delta Lake.

Competenze

• 3+ years of progressive experience in software, platform, DevOps/SRE, or data engineering.
• 2+ years building or operating cloud data platforms or production data systems.
• Strong Python skills for packaging, modular and testable code, and pytest.
• Experience building CI/CD pipelines (GitHub Actions preferred) for multi-environment deployments.
• Hands-on Terraform experience with Azure and Databricks resources.
• Production experience with Spark/PySpark and SQL.
• Experience with Delta Lake, Unity Catalog, Delta Live Tables, and ADLS Gen2.
• Experience supporting production systems in a DevOps/SRE or platform role.

Mansioni

• Platform Development: Build and maintain shared Python libraries and configuration-driven frameworks for onboarding pipelines.
• CI/CD & Infrastructure as Code: Own GitHub Actions pipelines and release processes for Databricks Asset Bundles and Terraform coverage.
• Operations & Reliability: Support production platform, improve observability, triage incidents, and develop runbooks and automation.
• Security & Governance: Maintain platform and data controls with Unity Catalog and data protection in healthcare.
• Domain Enablement: Partner with domain teams to onboard data sources and create self-service features.

Conoscenze

Python

CI/CD

Git-based code review

Clear communication

Formazione

Bachelor's degree or equivalent

Strumenti

GitHub Actions

Terraform

Azure Databricks

Delta Lake

Unity Catalog

Descrizione del lavoro

Position Summary

We are looking for a Senior Data Platform Engineer to help build, run, and evolve our Azure Databricks data platform, which ingests, validates, and protects data for the business domain data engineering teams.

Responsibilities

• Platform Development: Build and maintain the shared Python libraries and configuration-driven frameworks that let domain teams onboard and run their pipelines with minimal platform help.
• CI/CD & Infrastructure as Code: Own the GitHub Actions pipelines and release process that promote Databricks Asset Bundles from sandbox to production, and extend Terraform coverage of our Azure and Databricks infrastructure.
• Operations, Reliability & Recovery: Support the production platform, build enhanced observability, triage incidents, lead root cause analysis, and turn lessons learned into runbooks, automation, and permanent fixes. Track performance, reliability and delivery metrics from a data and platform perspective.
• Security & Governance: Maintain necessary platform and data controls with Unity Catalog, and support sensitive data protection and governance in a regulated healthcare environment.
• Domain Enablement: Partner with domain teams to onboard data sources, resolve escalated issues, and turn recurring needs into self-service features.

Required Experience

3+ years progressive in software, platform, DevOps/SRE, or data engineering, including:

• 2+ years building or operating cloud data platforms or production data systems.
• Software Engineering & CI/CD: Strong Python skills (packaging, modular and testable code, pytest), Git-based code review, and proven experience building CI/CD pipelines (GitHub Actions preferred) for multi-environment deployments.
• Infrastructure as Code: Hands-on Terraform experience managing Azure and Databricks resources.
• Data Engineering: Production experience with Spark/PySpark and SQL, including batch and incremental ingestion, schema evolution, medallion data modeling, and data quality validation.
• Databricks & Azure: Practical experience with Delta Lake, Unity Catalog, Jobs, and Delta Live Tables / Lakeflow (Asset Bundles and system tables a plus), and working knowledge of ADLS Gen2, Entra ID, Key Vault, and cloud networking.
• Operations: Experience supporting production systems in a DevOps, SRE, or platform role: monitoring and alerting, incident response, on-call, root cause analysis, runbooks, and disaster recovery planning.
• Communication: Clear communication and documentation, and the ability to work well on a remote team with technical and business stakeholders.

Preferred Qualifications

• 6+ years progressive in software, platform, DevOps/SRE, or data engineering,
• Disciplined use of AI to accelerate development and analysis, with the same engineering rigor and ownership of AI-assisted work as any other work shipped
• Familiarity with healthcare payer data (X12 834/837, HL7) and HIPAA; HITRUST experience a plus
• Databricks or Azure certifications

Education

Bachelor's degree or equivalent practical experience.

Anticipated Weekly Hours

40

Time Type

Full time

Pay Range

The typical pay range for this role is: $101,970.00 - $203,940.00This pay range represents the base hourly rate or base annual full-time salary for all positions in the job grade within which this position falls. The actual base salary offer will depend on a variety of factors including experience, education, geography and other relevant factors. This position is eligible for a CVS Health bonus, commission or short-term incentive program in addition to the base pay range listed above.

Benefits

Great benefits for great peopleWe take pride in offering a comprehensive and competitive mix of pay and benefits that reflects our commitment to our colleagues and their families.This full‑time position is eligible for a comprehensive benefits package designed to support the physical, emotional, and financial well‑being of colleagues and their families. The benefits for this position include medical, dental, and vision coverage, paid time off, retirement savings options, wellness programs, and other resources, based on eligibility.

Our people fuel our future. Our teams reflect the customers, patients, members and communities we serve and we are committed to fostering a workplace where every colleague feels valued and that they belong.

Additional details about available benefits are provided during the application process and on Benefits Moments.

We anticipate the application window for this opening will close on: 11/29/2026Qualified applicants with arrest or conviction records will be considered for employment in accordance with all federal, state and local laws.