Principal Observability Platform Architect

nscaleoperationsukltd; United States; Remote; USD 190,000 - 300,000; Full time

Benefits offered by this job

Medical coverage

Dental coverage

Vision coverage

Flexible PTO

Parental leave

Retirement plan

Job summary

nscaleoperationsukltd in the United States is seeking a principal-level observability engineer to set the technical direction for metrics, logs, traces, and alerting across large-scale infrastructure, accelerated computing clusters, and AI workloads. You will shape architecture and guide teams toward systems that remain clear and manageable as they grow.

This role demands deep hands-on experience with Prometheus, Grafana, OpenTelemetry, and modern data stores, plus strong Python or Go skills.

Qualifications

• Eight or more years in SRE, infrastructure, platform engineering, or observability-focused work.
• Experience operating observability infrastructure at substantial scale and designing for growth.
• Deep hands-on knowledge of several tools in the observability ecosystem (Prometheus, Grafana, OpenTelemetry, etc.).
• Strong programming skills in Python or Go with end-to-end ownership of complex systems.
• Kubernetes at scale and infrastructure-as-code using Terraform or Ansible.

Responsibilities

• Define platform architecture and long-term strategy for metrics, logs, traces, and alerting.
• Make durable decisions about data models, ingestion, retention, and high-cardinality data.
• Identify visibility gaps and design systems to improve incident detection and diagnosis.
• Partner with SRE, infrastructure, and AI/ML teams to embed observability into systems.
• Establish engineering patterns, mentor observability engineers, and review designs.
• Lead incident reviews and drive platform improvements from findings.
• Assess tools for signal quality, scale, and operational efficiency; retire unsuitable ones.

Skills

SRE

Platform engineering

Observability

Python

Go

Kubernetes

IaC

Tools

Prometheus

Thanos

VictoriaMetrics

Grafana

Loki

Tempo

OpenTelemetry

ClickHouse

Elastic

Job description

nscaleoperationsukltd in the United States is seeking a principal-level observability engineer to set the technical direction for metrics, logs, traces, and alerting across large-scale infrastructure, accelerated computing clusters, and AI workloads. You will shape architecture and guide teams toward systems that remain clear and manageable as they grow.

This role demands deep hands-on experience with Prometheus, Grafana, OpenTelemetry, and modern data stores, plus strong Python or Go skills.