Senior Observability Platform Engineer — GPU Infra

Devopsroles; Northern (KY); Hybrid; USD 160,000 - 230,000; Full time

Job summary

Nscale is seeking a Senior Observability Platform Engineer to design, build, and scale our observability platform for GPU clusters and AI workloads. You will balance usability, scalability, and reliability while embedding observability across services.

You'll work with SRE, infrastructure, and AI/ML teams to drive best practices, reduce noise, and improve incident response. This hands-on role offers influence over platform direction and evolution.

Qualifications

• 5+ years in SRE, infra engineering, platform engineering, or observability roles.
• Experience operating and scaling observability systems in production.
• Strong understanding of monitoring concepts: metrics, logs, traces, alerting, and SLOs.
• Hands-on with Prometheus, Thanos, VictoriaMetrics, Grafana, Loki, Tempo, OpenTelemetry, ClickHouse, Elastic.
• Solid programming skills (Python, Go, or similar).
• Experience with Kubernetes-based infrastructure.
• Familiarity with Infrastructure-as-Code (Terraform, Ansible, or similar).
• Pragmatic, reliable, maintainable approach and strong collaboration.

Responsibilities

• Design, build, and operate scalable observability systems across metrics, logs, traces, and alerting.
• Contribute to tooling, data pipelines, storage, and retention strategies.
• Improve signal quality by reducing noise and refining alerting practices.
• Identify observability gaps before reliability is impacted.
• Partner with SRE, infra, and AI/ML teams to embed observability.
• Develop reusable patterns, libraries, and best practices for consistency.
• Participate in incident response and postmortems to drive improvements.
• Evaluate and adopt tools to improve developer experience and scalability.
• Mentor engineers through code reviews and knowledge sharing.

Skills

SRE experience

Production observability

Python or Go

Cross-team collaboration

System design

Tools

Prometheus

Thanos

VictoriaMetrics

Grafana

Loki

Tempo

OpenTelemetry

ClickHouse

Elastic

Kubernetes

Job description

Nscale is seeking a Senior Observability Platform Engineer to design, build, and scale our observability platform for GPU clusters and AI workloads. You will balance usability, scalability, and reliability while embedding observability across services.

You'll work with SRE, infrastructure, and AI/ML teams to drive best practices, reduce noise, and improve incident response. This hands-on role offers influence over platform direction and evolution.