Staff Platform Data Engineer — Azure Databricks

CVS Health; Oregon; Hybrid; USD 130,000 - 261,000; Full time

Benefits offered by this job

Bonus eligibility

Equity awards

Comprehensive benefits

Job summary

CVS Health in Oregon seeks a Staff Data Engineer-Azure Databricks Platform to lead the development and operation of our Databricks-based data platform. You will architect and implement scalable data solutions, ensure reliability, performance and security, and mentor platform engineers.

This role partners with data teams and security teams to deliver cost-aware, compliant data pipelines and tooling, using Terraform and GitHub CI/CD, and guiding best practices across the organization.

Qualifications

• Experience: 8+ years of experience in a hands-on engineering role with a focus on cloud data platforms, distributed systems, and data engineering.
• Azure Databricks Expertise: Deep, hands-on experience with the Azure Databricks platform, including its administration, cluster management, job orchestration, and key features like Delta Lake and Unity Catalog.
• Cloud Proficiency: Expert-level knowledge of the Azure ecosystem, including Azure Data Lake Storage (ADLS), Azure Active Directory (AAD), and networking components.
• Programming & Scripting: Strong proficiency in Terraform, Python, Apache Spark/PySpark, SQL, and Shell scripting.
• Infrastructure as Code & CI/CD: Experience with Terraform and building automated CI/CD pipelines for platform and data applications.
• Distributed Systems: Understanding of distributed computing principles and large-scale data systems architecture.
• Problem-Solving: Strong analytical and problem-solving skills with a track record of diagnosing and resolving complex issues.
• Communication: Excellent communication, documentation, and presentation skills.

Responsibilities

• Operational Excellence: Oversee the operational health of the platform, monitoring, logging, and alerting; lead incident response and RCA for platform-wide problems.
• Platform Architecture & Strategy: Maintain and enhance Azure Databricks components; define the platform roadmap including Delta Lake, Unity Catalog, and GitHub CI/CD.
• Security & Governance: Enforce data security and governance, manage Unity Catalog for access control and data lineage.
• Performance & Cost Optimization: Develop strategies to optimize performance and minimize cloud spend.
• Tooling & Automation: Build IaC provisioning and CI/CD automation for Databricks workspaces and resources.
• Developer Enablement: Mentor data engineers, codify best practices, and create reusable patterns and libraries.
• Cross-Functional Collaboration: Work with data, security, networking, and infra teams to deliver a cohesive environment.
• Leading the Platform Engineers: Lead a team of 2 platform engineers.

Job description

CVS Health in Oregon seeks a Staff Data Engineer-Azure Databricks Platform to lead the development and operation of our Databricks-based data platform. You will architect and implement scalable data solutions, ensure reliability, performance and security, and mentor platform engineers.

This role partners with data teams and security teams to deliver cost-aware, compliant data pipelines and tooling, using Terraform and GitHub CI/CD, and guiding best practices across the organization.