SRE for AI Provider Operations: Resilient Routing

OpenRouter, Inc; New York (NY); Presencial; USD 140 000 - 190 000; Tempo integral

Resumo da oferta

OpenRouter is hiring its first AI Inference SRE to own the operational health of provider routes. You’ll ensure every endpoint is fast, correct, and available across 80+ providers, with telemetry, SLOs, and alerting that prevent customers from noticing issues.

You’ll lead on-call incident response, automate toil, and build tooling to detect degradations early. The role combines software engineering (TypeScript/Python) with understanding of LLM serving, streaming, and provider APIs to keep

Qualificações

• 4+ years in SRE, production engineering, or infra roles handling high-traffic systems.
• Strong observability: metrics, tracing, logs, SLOs and actionable alerts.
• Software engineer who writes tools, prefers TS/Python.
• Experience with distributed failure modes: timeouts, retries, backpressure.
• Calm incident commander who communicates under pressure.
• Understanding of how LLM inference is served: streaming, prompt caching, throughput tradeoffs.

Responsabilidades

• Provider health and observability: build monitoring for latency, throughput, error rates, uptime, and correctness.
• Detection and failover: improve detection of degraded endpoints and automatic traffic shifts.
• Incident response: own on-call, triage, mitigate, postmortems, and follow-ups.
• Provider accountability: turn telemetry into scorecards and SLO reporting.
• Quality regression detection: build canaries and evals to catch regressions.
• Automate the toil: replace manual provider-ops with safe tooling.
• Capacity and launch readiness: load-test endpoints before big launches.

Conhecimentos

SRE / production engineering

Observability

Typescript

Python

Distributed systems

Incident command

LLM inference serving

Ferramentas

Cloudflare Workers

Postgres

ClickHouse

GCP

Vercel

Descrição da oferta de emprego

OpenRouter is hiring its first AI Inference SRE to own the operational health of provider routes. You’ll ensure every endpoint is fast, correct, and available across 80+ providers, with telemetry, SLOs, and alerting that prevent customers from noticing issues.

You’ll lead on-call incident response, automate toil, and build tooling to detect degradations early. The role combines software engineering (TypeScript/Python) with understanding of LLM serving, streaming, and provider APIs to keep