Staff SRE – AI Gateway & Platform Reliability

Kraken; United States; À distance; USD 180 000 - 240 000; Plein temps

Résumé du poste

Kraken is seeking a senior-level Site Reliability Engineer to own the AI Gateway within the Dev Platform, ensuring reliable, scalable access to leading AI models. You will oversee SLOs, observability, and on-call practices across a production-critical service used throughout the company.

You will drive resilience through multi-provider routing, capacity planning, and automated deployments, while mentoring engineers to raise the overall reliability standard.

Qualifications

• Extensive experience in site reliability, infrastructure or platform engineering, with a track record of owning high-traffic, business-critical services in production.
• Deep hands-on experience with Kubernetes, a major cloud provider and infrastructure as code (for example Terraform).
• Strong command of SLOs, observability and incident management, and the judgment to know which reliability work matters most.
• Proficiency in a language such as Rust or Python, and comfort reading and contributing to a production service's codebase.
• Experience with API gateways, reverse proxies or traffic management: load balancing, rate limiting, failover and caching.
• Experience with Consul and Vault for service discovery, configuration and secrets management.
• Working knowledge of authentication and authorization mechanisms for internal platforms and APIs, such as OAuth 2.0/OIDC, JWT, mTLS and API keys, and how to apply them to a multi-tenant service.
• Experience with model evaluations: building or operating eval pipelines to compare models, prompts and agents, and to catch quality regressions before and after rollout.
• Familiarity with coding agents and agent harnesses (for example OpenHands, Claude Code or similar), including running them safely in sandboxed environments for engineering teams.
• Experience with agent memory systems and the databases behind them (vector stores, search, relational and key-value stores), and operating them reliably.
• Staff-level influence: you set direction across teams, write clear design documents and bring others with you.
• Good judgment balancing reliability, cost and developer velocity for internal customers.

Responsabilités

• Own the reliability, availability and performance of the AI Gateway as a production-critical service used across the company, including SLOs, error budgets and on-call practices
• Design the gateway's resilience: multi-provider routing, failover, retries, rate limiting and graceful degradation when an upstream model provider is slow or unavailable
• Set the technical direction for gateway architecture, capacity planning and scaling as LLM traffic grows and more workloads move onto the gateway
• Build observability for LLM traffic, covering latency, errors, token usage and cost, attributed by team and use case
• Partner with Security, Finance and AI governance stakeholders to put access controls, budgets, audit logging and data-handling requirements into the platform
• Lead incident response and blameless post-incident reviews, and turn what you learn into lasting reliability improvements
• Automate operations through infrastructure as code, safe deployments and progressive rollouts
• Mentor engineers and raise the operating standard across Dev Platform

Connaissances

Site reliability

Kubernetes

Cloud platforms

Terraform / IaC

SLOs / Observability

Incident management

Rust or Python

APIs gateways / traffic mgmt

Consul / Vault

Authn/Authz (OAuth2.0 / OIDC / JWT)

LLM model eval pipelines

Agent harnesses / sandboxing

Memory systems / vector stores

Cross-team influence

Cost reliability balance

Outils

Kubernetes

Terraform

OpenHands / Claude Code (examples)

Vector stores

Vault / Consul

Description du poste

Kraken is seeking a senior-level Site Reliability Engineer to own the AI Gateway within the Dev Platform, ensuring reliable, scalable access to leading AI models. You will oversee SLOs, observability, and on-call practices across a production-critical service used throughout the company.

You will drive resilience through multi-provider routing, capacity planning, and automated deployments, while mentoring engineers to raise the overall reliability standard.