Principal Architect: Observability & Hybrid Cloud

Hispanic Alliance for Career Enhancement

New Jersey

On-site

USD 144,200 - 288,400

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Medical, dental, vision coverage

Paid time off

Retirement options

Wellness programs

Job summary

CVS Health is seeking a Principal Architect to own observability and hybrid cloud architecture. You will shape reference architectures, telemetry standards, and instrumentation patterns across on‑prem OpenShift/KVM and multiple cloud platforms (Azure/AWS/GCP).

The role is hands‑on and will drive OpenTelemetry-first strategies with Grafana backends and integrations to incident tooling. You will mentor teams, define SLOs/SLIs, and ensure HIPAA/SOX/PCI alignment while collaborating with security

Qualifications

10+ years in infrastructure, cloud architecture, platform engineering, or SRE.
8+ years of architecture work in observability, cloud infrastructure, or both at a large enterprise.
5+ years with Kubernetes in production (OpenShift, EKS, AKS, or GKE).
5+ years with OpenTelemetry or similar frameworks (collectors, SDKs, semantic conventions, pipeline design).
5+ years with observability platforms: Grafana/Mimir/Loki/Tempo, Prometheus, Datadog, Splunk, Dynatrace, or comparable tools.
Experience defining SLOs/SLIs and building alerting strategies at an organizational level.
Proven track record writing architecture standards that other teams adopted and followed.
Able to communicate clearly with both engineers and senior leadership.

Responsibilities

Own the enterprise observability reference architecture covering metrics, logs, traces, and events across all environments (cloud and on‑prem).
Drive the OpenTelemetry‑first instrumentation strategy -- standard libraries, semantic conventions, collector topologies (DaemonSet, gateway, sidecar), and pipeline design.
Build and operate telemetry pipelines on Grafana Mimir, Loki, and Tempo, including multi‑tenant configurations, retention policies, and capacity planning.
Define how we measure reliability: SLOs, SLIs, error budgets, and alerting frameworks -- consistently across all lines of business.
Own the integration between observability tooling and incident management (ServiceNow ITOM, xMatters).
Drive telemetry schema standards to ensure teams emit data that is useful downstream, not just technically compliant.
Build and maintain reference architectures for our hybrid footprint: OpenShift on‑prem with KVM/libvirt and Dell PowerFlex storage, plus Azure, AWS, and GCP.
Lead standards work around workload identity and federation using SPIFFE/SPIRE and cloud‑native IAM patterns to move away from static secrets.
Provide guidance on compute runtime selection -- containers vs. VMs vs. bare metal vs. serverless -- with a clear decision framework for teams.
Help teams connect autoscaling and capacity planning behavior to actual telemetry signals.
Push FinOps maturity forward by integrating cost data into the observability stack, establishing unit economics, and working toward open billing standards like FOCUS.
Identify where AI/ML adds practical value in our observability stack -- anomaly detection, root cause analysis, log clustering, and smarter alerting.
Define observability standards for AI‑powered systems (agents, RAG pipelines) -- covering latency, token costs, model drift, and related signals.
Ensure new AI‑powered platforms are instrumented correctly from day one.
Participate in cross‑functional architecture working groups focused on observability and hybrid cloud standards.
Publish architecture decision records and reference implementations that teams can actually use.
Mentor architects and platform engineers; conduct architecture reviews to raise the bar across the org.
Work with security and compliance on HIPAA, SOX, and PCI requirements as they apply to telemetry and cloud infrastructure.
Represent CVS Health in vendor evaluations and stay connected to the open‑source ecosystem (CNCF, OpenTelemetry, Grafana Labs).