Senior Observability Platform Architect (Equity Eligible)

Nscale; Houston (TX); On-site; USD 190,000 - 300,000; Full time

Job summary

Nscale is seeking a Principal/Staff Observability Platform Engineer to own the technical direction of our observability platform for GPU clusters, AI workloads, and infrastructure. You’ll treat observability as a product, defining roadmaps and raising engineering standards across teams.

You’ll partner with SRE, infrastructure, and AI/ML groups to embed observability by design, lead incident postmortems, and mentor the team to build scalable, maintainable systems that reduce operational burden.

Qualifications

• 8+ years in SRE, infrastructure engineering, platform engineering, or observability-focused roles.
• Operated observability infrastructure at scale; design for 10x scale.
• Bias toward simplicity and avoid over-engineered stacks.
• Significant experience with Prometheus, Thanos, VictoriaMetrics, Grafana, Loki, Tempo, OpenTelemetry, ClickHouse, Elastic.
• Strong Python/Go or similar engineering fundamentals; end-to-end ownership.
• Kubernetes at scale; GPU/HPC (Slurm) familiarity is a strong plus.
• Architect systems, write code, review others, explain tradeoffs within week.
• Infrastructure-as-Code default: Terraform, Ansible, or equivalent.
• Influence without authority.

Responsibilities

• Own the technical strategy and architecture for observability across metrics, logs, traces, and alerting at scale.
• Drive platform decisions with multi-year impact: tooling, data models, ingestion patterns, retention, cardinality management.
• Identify systemic gaps before incidents; design platforms that make failure visible and fast to diagnose.
• Partner with SRE, infrastructure, and AI/ML teams to embed observability into how Nscale builds and operates.
• Define standards and patterns that engineers adopt, not by mandate.
• Mentor and grow the observability team; raise the ceiling of what the team can build.
• Lead incident postmortems and drive durable platform improvements.
• Evaluate and introduce tooling that meaningfully improves signal quality, operational efficiency, or scalability.

Skills

SRE experience

infrastructure experience

platform engineering

observability

leadership

Tools

Prometheus

Thanos

VictoriaMetrics

Grafana

Loki

Tempo

OpenTelemetry

ClickHouse

Elastic

Job description

Nscale is seeking a Principal/Staff Observability Platform Engineer to own the technical direction of our observability platform for GPU clusters, AI workloads, and infrastructure. You’ll treat observability as a product, defining roadmaps and raising engineering standards across teams.

You’ll partner with SRE, infrastructure, and AI/ML groups to embed observability by design, lead incident postmortems, and mentor the team to build scalable, maintainable systems that reduce operational burden.