Staff Engineer, Observability Platform & Telemetry

xAI

Palo Alto (CA)

On-site

USD 180,000 - 440,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity
Medical, vision, dental coverage
401(k) retirement plan
Disability insurance
Life insurance
Employee discounts

Job summary

xAI seeks a hands-on engineer to join a small, high-impact team building and maintaining the observability platform. You will own systems powering metrics, logs, tracing, and alerting to enable engineering teams to run services at scale, identify issues early, and drive systemic reliability improvements.

Qualifications include production-level Go/Rust/Scala, distributed systems expertise, and experience with Prometheus, Grafana, OpenTelemetry, VictoriaMetrics, or ClickHouse.

Qualifications

  • Production-level proficiency in Go, Rust, or Scala.
  • Deep understanding of distributed systems and telemetry architecture.
  • Experience building and operating infrastructure at scale.
  • Familiarity with observability stacks (Prometheus, Grafana, OpenTelemetry, VictoriaMetrics, ClickHouse).
  • Experience with Kafka, Redis, or large-scale time-series databases.
  • Experience operating observability pipelines in Kubernetes or similar environments.

Responsibilities

  • Design and implement scalable observability infrastructure for metrics, logging, and tracing.
  • Build high-performance telemetry pipelines that handle massive ingestion volumes.
  • Develop APIs, query engines, and UIs for real-time insights into services.
  • Define and enforce best practices for instrumentation, alerting, and reliability.
  • Partner with infra and product teams to deeply integrate observability into internal platforms.
  • Own the reliability, scalability, and performance of the observability stack end-to-end.

Skills

Go
Rust
Scala
Distributed systems
Observability
Prometheus
Grafana
OpenTelemetry
VictoriaMetrics
ClickHouse
Kafka
Redis
Kubernetes

Tools

Prometheus
Grafana
OpenTelemetry
VictoriaMetrics
ClickHouse

Job description

xAI seeks a hands-on engineer to join a small, high-impact team building and maintaining the observability platform. You will own systems powering metrics, logs, tracing, and alerting to enable engineering teams to run services at scale, identify issues early, and drive systemic reliability improvements.

Qualifications include production-level Go/Rust/Scala, distributed systems expertise, and experience with Prometheus, Grafana, OpenTelemetry, VictoriaMetrics, or ClickHouse.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Observability Engineer: Telemetry for AI Systems
Senior Observability Engineer: Telemetry for AI Systems

Anthropic Limited • Seattle (WA)

Hybrid
USD 320,000 - 485,000
Equity donation matching
Generous vacation & parental leave
Flexible working hours
+1
Observability Engineer: Scale Telemetry & AI Reliability
Observability Engineer: Scale Telemetry & AI Reliability

Baseten • New York (NY)

On-site
USD 165,000 - 330,000
Competitive compensation including 1x/
Full medical/dental/vision insurance
Flexible PTO with Winter Break
+3
Member of Technical Staff - Observability
Member of Technical Staff - Observability

xAI • Palo Alto (CA)

On-site
USD 180,000 - 440,000
Equity
Medical, vision, dental coverage
401(k) retirement plan
+3
Staff Backend Engineer, Observability (Remote)
Staff Backend Engineer, Observability (Remote)

Affirm, Inc. • United States

Remote
USD 204,000 - 290,000
Remote-friendly work setup
100% subsidized medical coverage for你/
Staff Observability Engineer: Telemetry at Scale
Staff Observability Engineer: Telemetry at Scale

Anthropic Limited • New York (NY)

Hybrid
USD 320,000 - 485,000
Senior Site Reliability Engineer - AI Observability & Infra
Senior Site Reliability Engineer - AI Observability & Infra

Tulip Interfaces • Boston (MA)

Hybrid
USD 160,000 - 200,000
Company equity
Flexible schedule
Health, dental, vision
Senior Observability & Profiling Engineer
Senior Observability & Profiling Engineer

Anthropic Limited • San Francisco (CA)

Hybrid
GBP 90,000 - 130,000
Senior Software Engineer (Observability)
Senior Software Engineer (Observability)

Coreweave • Sunnyvale (CA)

On-site
USD 180,000 - 240,000
Senior Staff Software Engineer, Observability (Remote)
Senior Staff Software Engineer, Observability (Remote)

Remote • San Francisco (CA)

Remote
USD 180,000 - 270,000
Comprehensive Healthcare Benefits and
401k with Employer Match
Global Benefit programs
Observability Engineer: Telemetry & Reliability (Equity)
Observability Engineer: Telemetry & Reliability (Equity)

Baseten • Seattle (WA)

On-site
USD 165,000 - 330,000
Competitive compensation
100% coverage of medical, dental, and视
Flexible PTO including Winter Break
+4