SRE: AI Inference Provider Operations

Speedrun Talent Network; United States; Hybrid; USD 120,000 - 160,000; Full time

Job summary

OpenRouter is hiring its first AI Inference SRE to own the operational health of provider supply. You’ll build monitoring, ensure endpoints are fast and reliable, and automate toil to scale with customer demand.

You will work on incident response, capacity planning, and provider accountability, partnering with the Provider Operations team to deliver high-availability infrastructure across 80+ providers and billions of requests daily.

Qualifications

• 4+ years in SRE, production engineering, or infra roles for high-traffic systems.
• Strong observability skills: metrics, tracing, logs, SLOs, actionable alerts.
• Capable software engineer preferring tools over runbooks; TypeScript and/or Python.
• Experience with distributed systems failure modes: timeouts, retries, backpressure.
• Calm incident commander with clear communication under pressure.
• Understand or eager to learn how LLM inference is served (streaming, tool calling).

Responsibilities

• Provider health and observability: monitor latency, throughput, uptime, and accuracy.
• Detection and failover: improve degraded endpoints detection and automatic routing.
• Incident response: on-call triage, mitigation, postmortems, follow-ups.
• Provider accountability: create scorecards and SLA reports for providers.
• Quality regression detection: canaries and evals for ML pipelines.
• Automate the toil: replace manual provider-ops with auditable tooling.
• Capacity and launch readiness: load-test endpoints before big launches.

Skills

SRE/infra

Observability

TypeScript

Python

Distributed systems

Incident response

Tools

Cloudflare Workers

Postgres

ClickHouse

GCP

Vercel

Job description

OpenRouter is hiring its first AI Inference SRE to own the operational health of provider supply. You’ll build monitoring, ensure endpoints are fast and reliable, and automate toil to scale with customer demand.

You will work on incident response, capacity planning, and provider accountability, partnering with the Provider Operations team to deliver high-availability infrastructure across 80+ providers and billions of requests daily.