AI Inference SRE — Provider Operations & Reliability

Devopsroles; United States; Remote; USD 170,000 - 230,000; Full time

Job summary

OpenRouter is accelerating enterprise AI adoption by operating the AI inference supply chain across 80+ providers. We are hiring our first AI Inference SRE to own the operational health of provider endpoints, ensuring they are fast, correct, and highly available, while detecting and routing around problems before customers notice.

You will join the Provider Operations team, report to the Provider Operations Manager, and build monitoring, automation, and fault-tolerance into our routing fabric.

Qualifications

• 4+ years in SRE, production engineering, or infrastructure roles.
• Observability tooling: metrics, tracing, logs, SLOs, actionable alerts.
• Capable software engineer who prefers writing tools over executing runbooks; TypeScript and/or Python.
• Experience with distributed systems failure modes: timeouts, retries, backpressure, partial outages.
• Calm, clear incident commander who communicates well under pressure.
• Knowledge of LLM inference serving: streaming, tool calls, prompt caching, throughput/latency tradeoffs.

Responsibilities

• Own provider health and observability: latency, throughput, error rates, uptime, correctness.
• Improve detection and failover; shift traffic away from degraded endpoints automatically.
• Own on-call for provider incidents: triage, mitigate, postmortems, follow-ups.
• Turn telemetry into scorecards and SLO reporting; be the escalation point for misbehaving endpoints.
• Build canaries and evals to catch regressions (quantization changes, tool calls, data issues).
• Automate manual provider-ops toil with auditable tooling.
• Build load-testing tooling to ensure readiness before big launches.

Skills

SRE experience

Observability tooling

TypeScript

Python

Distributed systems

Incident command

LLM inference knowledge

Tools

Cloudflare Workers

Postgres

ClickHouse

GCP

Vercel

Job description

OpenRouter is accelerating enterprise AI adoption by operating the AI inference supply chain across 80+ providers. We are hiring our first AI Inference SRE to own the operational health of provider endpoints, ensuring they are fast, correct, and highly available, while detecting and routing around problems before customers notice.

You will join the Provider Operations team, report to the Provider Operations Manager, and build monitoring, automation, and fault-tolerance into our routing fabric.