SRE, Provider Operations — AI Inference Reliability

AI Chopping Block, Inc.; Northern (KY); Hybrid; USD 120,000 - 160,000; Full time

Job summary

OpenRouter is hiring its first AI Inference SRE to own provider supply health and operational reliability. You will build monitoring for 80+ providers, implement rapid detection of degraded endpoints, and own on-call responses.

The role emphasizes tooling, SLOs, and scalable incident management across distributed systems. You should bring 4+ years in SRE/production engineering, strong observability skills, and experience with TypeScript or Python.

Qualifications

• 4+ years in SRE or production engineering for high-traffic systems.
• Strong with observability: metrics, tracing, logs, SLOs, alerting.
• Capable software engineer who writes tooling (TypeScript or Python).
• Experience with distributed systems failure modes: timeouts, retries, backpressure, partial outages.
• Calm incident commander with clear communication under pressure.
• Familiarity with LLM inference serving: streaming, tool calling, prompt caching.

Responsibilities

• Provider health and observability: Build and own monitoring for every provider and endpoint; set SLOs and alerts.
• Detection and failover: Improve detection and automatic traffic rerouting.
• Incident response: Own on-call, triage, mitigate, postmortems, follow-ups.
• Provider accountability: Create telemetry-based scorecards and escalation points.
• Quality regression detection: Build canaries and evals for regressions.
• Automate the toil: Replace manual ops with safe tooling.
• Capacity and launch readiness: Load-test endpoints before large launches.

Skills

SRE experience

Observability tooling

Distributed systems

Incident response

Software engineering

Tools

TypeScript

Python

Cloudflare Workers

Postgres

ClickHouse

GCP

Vercel

Job description

OpenRouter is hiring its first AI Inference SRE to own provider supply health and operational reliability. You will build monitoring for 80+ providers, implement rapid detection of degraded endpoints, and own on-call responses.

The role emphasizes tooling, SLOs, and scalable incident management across distributed systems. You should bring 4+ years in SRE/production engineering, strong observability skills, and experience with TypeScript or Python.