Senior Observability Platform Engineer - OpenTelemetry

CVS Health; Town of Florida (NY); Hybrid; USD 83,000 - 222,000; Full time

Benefits offered by this job

Benefits package

Job summary

CVS Health is seeking a Senior Observability Platform Engineer to design, build, and operate large-scale observability services that process billions of logs, metrics, and traces daily. You will develop backend services using Go, Python, and Java, and lead OpenTelemetry adoption across the enterprise.

You will partner with SRE, Cloud Engineering, and Infra teams to shape platform strategy, improve developer experience, and ensure secure, cost-efficient observability at scale.

Qualifications

• 5+ years of experience in Software Engineering.
• 3+ years of observability practices, including SLIs/SLOs/SLAs, alerting, and incident management.
• 3+ years building production-grade backend services in Go and/or Java.
• 3+ years implementing and operating OpenTelemetry, including OTLP, semantic conventions, and instrumentation patterns.
• 3+ years with cloud-native and containerized platforms (Docker, Kubernetes, Argo CD).
• 3+ years with public cloud platforms (AWS, GCP, or Azure).
• 2+ years designing and scaling distributed, high-volume data pipelines.
• 2+ years of Infrastructure as Code tools such as Terraform or CloudFormation.
• 2+ years with Helmcharts, Kustomize etc.
• 2+ years working with Grafana OSS or comparable observability backends (Grafana, Loki, Tempo, Mimir).
• 2+ years with relational databases (PostgreSQL, MySQL).

Responsibilities

• Develop custom software to drive the observability platform using Java Spring Boot, Python, Go.
• Design, build, and operate core observability platform services using Go, Python, Java (Spring Boot).
• Support and lead enterprise-wide adoption of OpenTelemetry, including client libraries and instrumentation patterns.
• Design, architect and scale high-throughput telemetry pipelines with focus on performance and cost efficiency.
• Develop self-service observability capabilities to simplify onboarding and troubleshooting.
• Implement end-to-end monitoring of the observability platform, defining SLOs and health checks.
• Collaborate with SRE, Platform, and Cloud teams to establish reliability standards.
• Participate in on-call rotations and lead incident mitigation and post-incident reviews.
• Automate operational workflows and improve CI/CD tooling and platform automation.
• Ensure secure telemetry pipelines through mTLS and secrets management.
• Produce and maintain high-quality technical documentation and standards.
• Engage with internal teams to influence roadmap and deliver platform improvements.
• Provide technical leadership through mentorship and architectural guidance.

Skills

Software engineering

Observability practices

OpenTelemetry

Cross-team collaboration

Incident management

Education

Bachelor's degree

Tools

Go

Java

Python

Docker

Kubernetes

Argo CD

AWS

GCP

Azure

Terraform

CloudFormation

Helm

Kustomize

Grafana

PostgreSQL

MySQL

Envoy

Istio

Datadog

New Relic

AppDynamics

OpsGenie

PagerDuty

Kafka

Pulsar

Job description

CVS Health is seeking a Senior Observability Platform Engineer to design, build, and operate large-scale observability services that process billions of logs, metrics, and traces daily. You will develop backend services using Go, Python, and Java, and lead OpenTelemetry adoption across the enterprise.

You will partner with SRE, Cloud Engineering, and Infra teams to shape platform strategy, improve developer experience, and ensure secure, cost-efficient observability at scale.