Senior Data Engineer - Observability & Databricks

Koitecc Solutions; Arizona; On-site; USD 93,000 - 222,000; Full time

Job summary

CVS Health is seeking a Senior Data Engineer, Observability Engineering to join the Enterprise Observability Platform. You will design, build, and operate scalable Databricks-based data pipelines that ingest telemetry, infrastructure, and security data.

You will optimize PySpark workloads and lakehouse architectures (bronze/silver/gold), enforce governance, metadata management, and data quality, and collaborate with Security, Infrastructure, Observability, and Platform teams.

Qualifications

• 5+ years building enterprise data engineering solutions using Python, Spark, PySpark, SQL.
• 4+ years with Databricks in production, including Delta Lake and Unity Catalog.
• 3+ years delivering high-volume batch/streaming data solutions with data quality controls.

Responsibilities

• Design, build, deploy, and support scalable Databricks-based data pipelines ingesting telemetry, infrastructure, and security data.
• Develop and optimize PySpark workloads and lakehouse architectures (bronze/silver/gold) with a focus on performance and reliability.
• Collaborate with Security, Infrastructure, Observability, and Platform Engineering teams to onboard data sources, govern data, and deliver trusted enterprise data products.
• Contribute to engineering standards, code reviews, CI/CD practices, monitoring, and platform optimization to reduce technical debt.

Skills

Python

Spark

PySpark

SQL

Data pipelines

Education

Bachelor's degree

Tools

Databricks

Delta Lake

Unity Catalog

GitHub Actions

Azure DevOps

Job description

CVS Health is seeking a Senior Data Engineer, Observability Engineering to join the Enterprise Observability Platform. You will design, build, and operate scalable Databricks-based data pipelines that ingest telemetry, infrastructure, and security data.

You will optimize PySpark workloads and lakehouse architectures (bronze/silver/gold), enforce governance, metadata management, and data quality, and collaborate with Security, Infrastructure, Observability, and Platform teams.