Observability Engineer: Scalable Telemetry & SLOs

Triwill Group; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Competitive compensation with equity

Comprehensive health insurance

Flexible PTO including Winter Break

Paid parental leave

Fertility and family-building stipend

401(k)

Learning and networking opportunities

Job summary

Baseten is seeking talented Software Engineers to join our Observability team within the Infrastructure organization. As an early member, you will help build the observability experience for internal and external customers, improving reliability of Baseten's production systems.

You will design scalable telemetry ingest and storage pipelines across multi-cloud infrastructure, build instrumentation libraries, and drive SLOs with low noise. Collaboration with product and infra teams is essential.

Qualifications

• Have deep experience with at least one observability signal area (metrics, logging, tracing, or error analytics) and familiarity with the others.
• Understand high-throughput data pipelines, columnar storage engines, and the tradeoffs involved in ingesting and querying telemetry data at scale.
• Have experience operating or building on top of observability platforms such as Prometheus, Grafana, ClickHouse, OpenTelemetry, or similar systems.
• Have strong proficiency in at least one of Python, Rust, or Go.
• Have excellent communication skills and enjoy partnering with internal teams to improve their operational visibility and incident response capabilities.
• Interest in applying AI/LLMs to operational workflows such as automated root cause analysis, anomaly detection, or intelligent alerting.

Responsibilities

• Design and build scalable telemetry ingest and storage pipelines for metrics, logs, and traces across Baseten’s multi-cloud infrastructure
• Own and evolve core observability platforms, driving migrations and architectural improvements that improve reliability, reduce cost, and scale with organizational growth
• Build instrumentation libraries, SDKs, and integrations that make it easy for engineering teams to emit high-quality telemetry from their services
• Drive alerting and SLO infrastructure that enables teams to define, monitor, and respond to reliability targets with minimal noise
• Partner with Inference, Product, and Infrastructure teams to ensure observability solutions meet the unique needs of each organization

Skills

Python

Rust

Go

Communication

Tools

Prometheus

Grafana

OpenTelemetry

ClickHouse

Job description

Baseten is seeking talented Software Engineers to join our Observability team within the Infrastructure organization. As an early member, you will help build the observability experience for internal and external customers, improving reliability of Baseten's production systems.

You will design scalable telemetry ingest and storage pipelines across multi-cloud infrastructure, build instrumentation libraries, and drive SLOs with low noise. Collaboration with product and infra teams is essential.