Senior Data Engineer, Observability Platform

Koitecc Solutions; Massachusetts; Hybrid; USD 93,000 - 222,000; Full time

Benefits offered by this job

Medical, dental, vision coverage

Paid time off

Retirement savings options

Wellness programs

Job summary

CVS Health is seeking a highly skilled Senior Data Engineer, Observability Engineering to join the Enterprise Observability Platform and help advance data capabilities for observability, infrastructure, and security analytics across the organization.

This role focuses on Databricks-based data engineering, building scalable pipelines, maintaining data quality, and partnering with security and platform teams to deliver trusted enterprise data products at scale.

Qualifications

• 5+ years of experience building and supporting enterprise data engineering solutions using Python, Spark, PySpark, SQL, and distributed data processing technologies.
• 5+ years designing, developing, and operating large-scale data pipelines and lakehouse architectures across multiple data sources.
• 4+ years working with Databricks in production environments, including Delta Lake, Unity Catalog, and platform operations.
• 3+ years developing high-volume batch and streaming data solutions with data quality controls and reliability improvements.
• 2+ years collaborating with engineering teams, applying cloud security and access management principles.

Responsibilities

• Design, build, deploy, and support Databricks-based data pipelines and lakehouse solutions for observability, infrastructure, and security data.
• Develop and optimize PySpark workloads and Structured Streaming with bronze/silver/gold data architectures.
• Partner with Security, Infrastructure, Observability, and Platform Engineering to onboard data sources and enforce governance standards.
• Contribute to engineering standards, code reviews, CI/CD practices, monitoring strategies, and platform optimization.
• Promote best practices for Databricks, Delta Lake, Unity Catalog, and cost-efficient data platform operations.

Skills

Python

Spark / PySpark

SQL

Distributed data processing

Databricks

Delta Lake

Unity Catalog

Structured Streaming

Data governance

Cloud security / IAM

Education

Bachelor's degree or higher in Computer Science or related field

Tools

Databricks

Delta Lake

Unity Catalog

GitHub Actions

Azure DevOps

Job description

CVS Health is seeking a highly skilled Senior Data Engineer, Observability Engineering to join the Enterprise Observability Platform and help advance data capabilities for observability, infrastructure, and security analytics across the organization.

This role focuses on Databricks-based data engineering, building scalable pipelines, maintaining data quality, and partnering with security and platform teams to deliver trusted enterprise data products at scale.