Observability Engineer: Scale Telemetry & AI Reliability

Baseten; New York (NY); On-site; USD 165,000 - 330,000; Full time

Benefits offered by this job

Competitive compensation including 1x/

Full medical/dental/vision insurance

Flexible PTO with Winter Break

Paid parental leave

401(k)

Learning & networking opportunities

Job summary

Baseten is hiring for a Software Engineer on the Observability team to shape the telemetry experience across multi-cloud infrastructure. You will build scalable ingest pipelines, storage, and tooling to detect and resolve issues quickly, enabling reliable production systems.

You will work with Prometheus, Grafana, OpenTelemetry, ClickHouse, and modern languages like Python/Go/Rust, partnering with cross-functional teams to deliver high-impact diagnostics and alerting capabilities.

Qualifications

• Deep experience with at least one observability signal area (metrics, logging, tracing, or error analytics) and familiarity with the others.
• Experience operating or building on top of observability platforms such as Prometheus, Grafana, ClickHouse, OpenTelemetry, or similar systems.
• Strong proficiency in Python, Rust, or Go.

Responsibilities

• Design and build scalable telemetry ingest and storage pipelines for metrics, logs, and traces across Baseten’s multi-cloud infrastructure.
• Own and evolve core observability platforms, driving migrations and architectural improvements that improve reliability, reduce cost, and scale with organizational growth.
• Build instrumentation libraries, SDKs, and integrations that make it easy for engineering teams to emit high-quality telemetry from their services.
• Drive alerting and SLO infrastructure that enables teams to define, monitor, and respond to reliability targets with minimal noise.
• Partner with Inference, Product, and Infrastructure teams to ensure observability solutions meet the unique needs of each organization.

Skills

Prometheus

Grafana

OpenTelemetry

Python

Go

Rust

Telemetry pipelines

Observability

Tools

ClickHouse

Telemetry SDKs

Job description

Baseten is hiring for a Software Engineer on the Observability team to shape the telemetry experience across multi-cloud infrastructure. You will build scalable ingest pipelines, storage, and tooling to detect and resolve issues quickly, enabling reliable production systems.

You will work with Prometheus, Grafana, OpenTelemetry, ClickHouse, and modern languages like Python/Go/Rust, partnering with cross-functional teams to deliver high-impact diagnostics and alerting capabilities.