Senior Observability Platform Engineer - Scale & Reliability

CVS Health; West Virginia; Hybrid; USD 83,000 - 222,000; Full time

Job summary

CVS Health Enterprise Technology is hiring a Senior Observability Platform Engineer to design, build, and operate large‑scale observability services that process billions of logs, metrics, and traces daily. You will lead OpenTelemetry adoption and drive a self‑service observability ecosystem across SRE, Cloud, and application teams.

You will mentor engineers, influence architecture, and deliver reliable, cost‑efficient instrumentation across on‑prem, hybrid, and multiple public clouds, ensuring

Qualifications

• 5+ years of experience in Software Engineering.
• 3+ years of experience with observability practices, including SLIs/SLOs/SLAs, alerting, and incident management.
• 3+ years building production‑grade backend services in Go and/or Java.
• 3+ years implementing and operating OpenTelemetry, including OTLP, semantic conventions, and instrumentation patterns.
• 3+ years with cloud‑native and containerized platforms (Docker, Kubernetes, Argo CD).
• 3+ years working with public cloud platforms (AWS, GCP, or Azure).
• 2+ years designing and scaling distributed, high‑volume data pipelines.
• 2+ years of experience with Infrastructure as Code tools such as Terraform or CloudFormation.
• 2+ years of experience with Helmcharts, Kustomize etc.
• 2+ years working with Grafana OSS or comparable observability backends (e.g., Grafana, Loki, Tempo, Mimir).
• 2+ years with relational databases (PostgreSQL, MySQL).

Responsibilities

• Develop custom software to drive the observability platform using Java Spring Boot, Python, Go‑lang etc.
• Design, build, and operate core observability platform services using Go, Python, Java (Spring Boot)
• Support and lead enterprise‑wide adoption of OpenTelemetry, including client libraries, semantic conventions, instrumentation patterns, and Collector/agent strategy
• Design, architect and scale high‑throughput, fault‑tolerant telemetry pipelines (logs, metrics, traces) with a focus on performance, reliability, and cost efficiency
• Develop self‑service observability capabilities that simplify onboarding, troubleshooting, and adoption for application teams
• Implement end‑to‑end monitoring of the observability platform itself, defining SLOs, health checks, and alerting
• Collaborate with SRE, Platform, and Cloud teams to establish reliability standards, error budgets, and incident response practices
• Participate in on‑call rotations and lead incident mitigation, root‑cause analysis, and post‑incident reviews
• Automate operational workflows and eliminate manual toil through tooling, CI/CD enhancements, and platform automation
• Ensure secure telemetry pipelines through mTLS, secrets management, and zero‑trust design patterns
• Produce and maintain high‑quality technical documentation, standards, and best practices
• Engage with internal engineering teams to gather requirements, influence roadmap prioritization, and deliver platform improvements
• Provide technical leadership through mentorship, design reviews, architectural guidance, and cross‑team collaboration with principal engineers and engineering leadership

Skills

Software Engineering

Observability

Go

Python

Java

OpenTelemetry

Kubernetes

AWS/GCP/Azure

CI/CD

Infrastructure as Code

Terraform

CloudFormation

Grafana

PostgreSQL/MySQL

Education

Bachelor's degree or equivalent work experience

Tools

Docker

Kubernetes

Argo CD

Terraform

CloudFormation

Helm

Kustomize

Grafana

Job description

CVS Health Enterprise Technology is hiring a Senior Observability Platform Engineer to design, build, and operate large‑scale observability services that process billions of logs, metrics, and traces daily. You will lead OpenTelemetry adoption and drive a self‑service observability ecosystem across SRE, Cloud, and application teams.

You will mentor engineers, influence architecture, and deliver reliable, cost‑efficient instrumentation across on‑prem, hybrid, and multiple public clouds, ensuring