Staff Observability Engineer (Remote) — SRE Leadership

PsiQuantum; United States; Remote; USD 180,000 - 260,000; Full time

Job summary

PsiQuantum in Palo Alto, CA (remote) is seeking a Staff Software Engineer for the Observability team. You will wire up OpenTelemetry instrumentation, scale Prometheus and Grafana, and turn raw signals into trusted dashboards and alerts that guide production decisions.

This hands-on role combines design and operation across on-premises Kubernetes, cloud, and hardware-in-the-loop systems. You will lead architecture, own major components of the observability stack, participate in on-call rotation,

Qualifications

• 8+ years building and operating observability systems in production.
• Bachelor's or Master's degree in CS/Engineering/Physics or related.
• Deep Prometheus and Grafana experience at scale.
• Production OpenTelemetry including Collector, OTLP, and instrumentation.
• Kubernetes on-prem and cloud with Terraform; GitLab CI preferred.
• Python and Go or Rust; production exporters and collectors.
• Linux, networking basics; debugging distributed systems.
• Experience on-call and post-incident reviews.
• Staff-level technical leadership and cross-team influence.
• Familiar with AI/LLM tooling, privacy, and secure coding practices.

Responsibilities

• Design, deploy, and operate the observability platform across on-premises and cloud environments.
• Roll out OpenTelemetry across the fleet and guide service teams in instrumentation.
• Run Prometheus in production, including discovery, rules, Alertmanager, and storage.

Skills

Observability

Prometheus

Grafana

OpenTelemetry

Python

Go

Rust

Kubernetes

Linux

Incident response

Leadership

AI/LLM tooling

Education

Bachelor's or Master's degree in CS/Engineering/Physics or related

Tools

Terraform

GitLab CI

OTLP

Job description

PsiQuantum in Palo Alto, CA (remote) is seeking a Staff Software Engineer for the Observability team. You will wire up OpenTelemetry instrumentation, scale Prometheus and Grafana, and turn raw signals into trusted dashboards and alerts that guide production decisions.

This hands-on role combines design and operation across on-premises Kubernetes, cloud, and hardware-in-the-loop systems. You will lead architecture, own major components of the observability stack, participate in on-call rotation,