Member of Technical Staff - Observability

xAI

Palo Alto (CA)

On-site

USD 180,000 - 440,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity
Medical, vision, dental coverage
401(k) retirement plan
Disability insurance
Life insurance
Employee discounts

Job summary

xAI seeks a hands-on engineer to join a small, high-impact team building and maintaining the observability platform. You will own systems powering metrics, logs, tracing, and alerting to enable engineering teams to run services at scale, identify issues early, and drive systemic reliability improvements.

Qualifications include production-level Go/Rust/Scala, distributed systems expertise, and experience with Prometheus, Grafana, OpenTelemetry, VictoriaMetrics, or ClickHouse.

Qualifications

  • Production-level proficiency in Go, Rust, or Scala.
  • Deep understanding of distributed systems and telemetry architecture.
  • Experience building and operating infrastructure at scale.
  • Familiarity with observability stacks (Prometheus, Grafana, OpenTelemetry, VictoriaMetrics, ClickHouse).
  • Experience with Kafka, Redis, or large-scale time-series databases.
  • Experience operating observability pipelines in Kubernetes or similar environments.

Responsibilities

  • Design and implement scalable observability infrastructure for metrics, logging, and tracing.
  • Build high-performance telemetry pipelines that handle massive ingestion volumes.
  • Develop APIs, query engines, and UIs for real-time insights into services.
  • Define and enforce best practices for instrumentation, alerting, and reliability.
  • Partner with infra and product teams to deeply integrate observability into internal platforms.
  • Own the reliability, scalability, and performance of the observability stack end-to-end.

Skills

Go
Rust
Scala
Distributed systems
Observability
Prometheus
Grafana
OpenTelemetry
VictoriaMetrics
ClickHouse
Kafka
Redis
Kubernetes

Tools

Prometheus
Grafana
OpenTelemetry
VictoriaMetrics
ClickHouse

Job description

ABOUT xAI

xAI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company's mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All employees are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates.

ABOUT THE ROLE:

You will be part of the small, high-impact team responsible for building and maintaining X's observability platform. You'll own critical systems that power metrics, logs, tracing, and alerting enabling engineering teams to operate services at scale, identify issues before they impact users, and drive systemic reliability improvements.

RESPONSIBILITIES:
  • Design and implement scalable observability infrastructure for metrics, logging, and tracing.
  • Build high-performance telemetry pipelines that handle massive ingestion volumes.
  • Develop APIs, query engines, and UIs that allow engineers to get real-time insights into their services.
  • Define and enforce best practices for instrumentation, alerting, and reliability across the company.
  • Partner with infrastructure and product teams to deeply integrate observability into our internal platforms.
  • Own the reliability, scalability, and performance of the observability stack end-to-end.
BASIC QUALIFICATIONS:
  • Production-level proficiency in Go, Rust, Scala, or a similar languages
  • Deep understanding of distributed systems and telemetry architecture.
  • Experience building and operating infrastructure at scale.
  • Familiarity with observability stacks such as Prometheus, Grafana, OpenTelemetry, VictoriaMetrics, or ClickHouse.
  • Experience with Kafka, Redis, or large-scale time series databases.
  • Experience operating observability pipelines in Kubernetes or similar orchestration environments.
COMPENSATION AND BENEFITS:

$180,000 - $440,000 USD

Base salary is just one part of our total rewards package at xAI, which also includes equity, comprehensive medical, vision, and dental coverage, access to a 401(k) retirement plan, short & long-term disability insurance, life insurance, and various other discounts and perks.

xAI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice .

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer - X Data
Software Engineer - X Data

xAI • Palo Alto (CA)

On-site
USD 125,000 - 400,000
Equity
Medical coverage
Vision coverage
+5
Sr. IT Data Engineer
Sr. IT Data Engineer

xAI • Palo Alto (CA)

On-site
USD 135,000 - 260,000
Equity
Medical, vision & dental
401(k) plan
+3
Sr. IT Data Engineer
Sr. IT Data Engineer

Maven Ventures • Palo Alto (CA)

On-site
USD 135,000 - 260,000
Equity
Comprehensive medical, vision, and dental coverage
401(k) retirement plan
+2
Member of Technical Staff - Mid-training
Member of Technical Staff - Mid-training

SpaceXAI • Palo Alto (CA)

On-site
USD 180,000 - 440,000
Member of Technical Staff - Real-Time Storage
Member of Technical Staff - Real-Time Storage

xAI • Palo Alto (CA)

On-site
USD 180,000 - 440,000
Equity
Comprehensive medical, vision, and 
d
401(k) retirement plan
+3
Member of Technical Staff - Ads
Member of Technical Staff - Ads

xAI • Palo Alto (CA)

On-site
USD 180,000 - 440,000
Equity
Medical coverage
Vision coverage
+5
Member of Technical Staff - Post-Training and RL
Member of Technical Staff - Post-Training and RL

xAI • Palo Alto (CA)

On-site
USD 180,000 - 600,000
Equity
Medical, Vision, and Dental coverage
401(k) retirement plan
+3
Engineering Technical Lead Manager (TLM) - Enterprise
Engineering Technical Lead Manager (TLM) - Enterprise

xAI • Palo Alto (CA)

On-site
USD 180,000 - 440,000
Comprehensive medical, vision, and dental coverage
401(k) retirement plan
Equity
Member of Technical Staff - Post-Training and RL
Member of Technical Staff - Post-Training and RL

SpaceXAI • Palo Alto (CA)

On-site
USD 180,000 - 600,000
Equity
Medical coverage
Dental coverage
+5
Staff Engineer, Observability Platform & Telemetry
Staff Engineer, Observability Platform & Telemetry

xAI • Palo Alto (CA)

On-site
USD 180,000 - 440,000
Equity
Medical, vision, dental coverage
401(k) retirement plan
+3