Site Reliability Engineer, Provider Operations OpenRouter · Remote · US · Site Reliability $245,000–$295,000 5d ago

Devopsroles; United States; Remote; USD 170,000 - 230,000; Full time

Job summary

OpenRouter is accelerating enterprise AI adoption by operating the AI inference supply chain across 80+ providers. We are hiring our first AI Inference SRE to own the operational health of provider endpoints, ensuring they are fast, correct, and highly available, while detecting and routing around problems before customers notice.

You will join the Provider Operations team, report to the Provider Operations Manager, and build monitoring, automation, and fault-tolerance into our routing fabric.

Qualifications

• 4+ years in SRE, production engineering, or infrastructure roles.
• Observability tooling: metrics, tracing, logs, SLOs, actionable alerts.
• Capable software engineer who prefers writing tools over executing runbooks; TypeScript and/or Python.
• Experience with distributed systems failure modes: timeouts, retries, backpressure, partial outages.
• Calm, clear incident commander who communicates well under pressure.
• Knowledge of LLM inference serving: streaming, tool calls, prompt caching, throughput/latency tradeoffs.

Responsibilities

• Own provider health and observability: latency, throughput, error rates, uptime, correctness.
• Improve detection and failover; shift traffic away from degraded endpoints automatically.
• Own on-call for provider incidents: triage, mitigate, postmortems, follow-ups.
• Turn telemetry into scorecards and SLO reporting; be the escalation point for misbehaving endpoints.
• Build canaries and evals to catch regressions (quantization changes, tool calls, data issues).
• Automate manual provider-ops toil with auditable tooling.
• Build load-testing tooling to ensure readiness before big launches.

Skills

SRE experience

Observability tooling

TypeScript

Python

Distributed systems

Incident command

LLM inference knowledge

Tools

Cloudflare Workers

Postgres

ClickHouse

GCP

Vercel

Job description

About OpenRouter

OpenRouter is the leading AI routing and infrastructure layer that enterprises use to access, manage, and optimize the best large language models across providers—without lock-in, capacity constraints, or unnecessary cost. We power the most advanced AI teams in the world by giving them the flexibility to move fast, scale confidently, and stay future-proof as models evolve.

As enterprise adoption of AI accelerates, OpenRouter sits at the center of how organizations operationalize LLMs across research, product, and production workloads.

About the Role

OpenRouter routes almost a billion requests and more than 20 trillion tokens a day, across 80+ providers and thousands of endpoints. Every one of those providers can degrade, rate-limit, change behavior, or go down without warning. Our customers count on us to absorb that chaos so their apps never notice.

We're hiring our first AI Inference SRE to own the operational health of our provider supply. You'll make sure every endpoint we route to is fast, correct, and available, and that we detect and route around problems before customers do. You'll sit on the Provider Operations team, reporting to the Provider Operations Manager.

What You'll Do

• Provider health and observability. Build and own monitoring for every provider and endpoint: latency, throughput, error rates, uptime, and output correctness. Set SLOs per provider tier and alert on them.

• Detection and failover. Improve how quickly we detect degraded endpoints, and work with the routing team so traffic shifts away from them automatically.

• Incident response. Own on-call for provider incidents: triage, mitigate, communicate with providers, run postmortems, and drive follow-ups to closure.

• Provider accountability. Turn telemetry into scorecards and SLO reporting that providers act on, and be the technical escalation point when a provider's endpoint is misbehaving.

• Quality regression detection. Build continuous canaries and evals that catch silent regressions (quantization changes, broken tool calling, truncated streams, pricing or usage-reporting mismatches), not just outright downtime.

• Automate the toil. Replace manual provider-ops work (disabling endpoints, capacity changes, deprecations, rate-limit tuning) with safe, auditable tooling.

• Capacity and launch readiness. Build tooling to load-test endpoints before big launches so day-zero traffic doesn't take them down.

About You

• 4+ years in SRE, production engineering, or infrastructure roles running high-traffic, customer-facing systems.

• Strong with observability tooling and practice: metrics, tracing, logs, SLOs/error budgets, alerting that is always actionable.

• Capable software engineer who prefers writing tools over executing runbooks. TypeScript and/or Python.

• Experienced with distributed systems failure modes: timeouts, retries, backpressure, partial outages, noisy neighbors.

• Calm, clear incident commander who communicates well with external partners under pressure.

• Understands, or is eager to learn deeply, how LLM inference is served: streaming, tool calling, prompt caching, throughput/latency tradeoffs, and how provider APIs differ.

Nice to Have

• Experience at an inference provider, model lab, GPU cloud, or API gateway/CDN company.

• Experience with our stack: TypeScript, Cloudflare Workers, Postgres, ClickHouse, GCP, Vercel.

• Background in routing, load balancing, or traffic management systems.

• Experience with evals or synthetic monitoring for ML systems.