Staff Observability Engineer — SRE for Quantum Infra

PSI Quantum; Palo Alto (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

PsiQuantum is seeking a Staff Software Engineer for the Observability team to own and evolve the telemetry platform across on‑prem Kubernetes and cloud. You will wire up OpenTelemetry, scale Prometheus and Grafana, and translate raw signals into trusted dashboards and alerts.

You will collaborate with hardware teams to instrument devices and ensure reliable data flow into the platform. You will lead design reviews, mentor engineers, and help set conventions for instrumenting systems, while

Qualifications

• Staff-level observability experience with ownership of stack design and operation.
• Hands-on instrumentation of devices and telemetry in Kubernetes and cloud environments.
• Ability to collaborate with hardware teams to collect reliable telemetry data.

Responsibilities

• Design, deploy, and operate the observability platform across on‑prem Kubernetes and cloud.
• Roll out OpenTelemetry, design telemetry pipelines, and guide SDK instrumentation in multiple languages.
• Run Prometheus in production: service discovery, rules, alerts, and long‑term storage.
• Manage Grafana dashboards and data sources as code across environments.
• Instrument custom devices and lab equipment to feed telemetry into the platform.
• Define telemetry standards for naming, cardinality, logging, and tracing.
• Partner with service owners to define SLOs and actionable alerts; reduce noise.
• Lead on‑call rotations and post‑incident reviews; drive fixes and improvements.
• Operate the platform as a production service with CI/CD and infrastructure as code.
• Apply AI/LLM tooling responsibly to accelerate engineering while maintaining security.

Skills

OpenTelemetry

Prometheus

Grafana

Python

Rust

Kubernetes

GitLab CI/CD

SLOs

Incident response

LLM tooling

Tools

OpenTelemetry

Prometheus

Grafana

Python

Rust

Kubernetes

GitLab CI/CD

Job description

PsiQuantum is seeking a Staff Software Engineer for the Observability team to own and evolve the telemetry platform across on‑prem Kubernetes and cloud. You will wire up OpenTelemetry, scale Prometheus and Grafana, and translate raw signals into trusted dashboards and alerts.

You will collaborate with hardware teams to instrument devices and ensure reliable data flow into the platform. You will lead design reviews, mentor engineers, and help set conventions for instrumenting systems, while