Staff Site Reliability Engineer - Dev Platform

kraken.com; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Kraken is seeking a lead reliability engineer to oversee an internal language-model gateway used by engineering teams to access multiple model providers.

You will fuse production operations, platform design, and technical leadership to keep services dependable as usage grows, balancing resilience, security, cost, and developer needs.

Qualifications

• Extensive experience operating business-critical production services in SRE, infra, or platform teams.
• Hands-on with Kubernetes, a major cloud platform, and Terraform for IaC.
• Strong knowledge of reliability, observability, and incident management.

Responsibilities

• Own service availability, performance, and on-call practices with SLOs and error budgets.
• Design routing, retries, rate limits, failover, and degradation for provider disruptions.
• Set direction for gateway architecture, capacity planning, and scaling.
• Build observability for latency, errors, token usage, and cost by workload.
• Collaborate with security, finance, and governance on access controls and logs.
• Lead incident response, conduct blameless reviews, and drive reliability improvements.
• Automate infrastructure and deployments with code and progressive releases.

Skills

Site reliability

Infrastructure

Platform engineering

Kubernetes

Cloud platforms

Terraform

Rust/Python

API gateways

Auth methods

Leadership

Tools

Kubernetes

Terraform

Consul

Vault

OIDC

JWT

Job description

Role overview

Lead reliability engineering for an internal language-model gateway used by engineering teams to access multiple model providers. The role combines production operations, platform architecture, and technical leadership: you’ll help the service remain dependable as usage grows while balancing resilience, security, cost, and developer needs.

Responsibilities

• Own service availability and performance, including service-level objectives, error budgets, and on-call practices.
• Design routing, retries, rate limits, failover, and graceful degradation for upstream provider disruptions.
• Set direction for gateway architecture, capacity planning, and scaling.
• Build observability for latency, errors, token usage, and cost, with useful attribution by team and workload.
• Work with security, finance, and governance partners on access controls, budgets, audit logs, and data handling.
• Lead incident response and blameless reviews, turn findings into reliability improvements, and mentor engineers.
• Automate infrastructure and operations through code, safe deployments, and progressive releases.

Requirements

• Extensive experience operating business-critical production services in site reliability, infrastructure, or platform engineering.
• Hands-on experience with Kubernetes, a major cloud platform, and infrastructure-as-code tools such as Terraform.
• Strong practical knowledge of reliability objectives, observability, and incident management.
• Proficiency in Rust or Python, with the ability to work in a production codebase.
• Experience with API gateways, reverse proxies, or traffic management, including load balancing, rate limiting, and failover.
• Familiarity with Consul and Vault, and with API authentication and authorization such as OAuth/OIDC, JWT, mutual TLS, or API keys.
• Staff-level technical leadership: setting direction across teams, writing design documents, and bringing others along.

Nice to have

• Experience with LLM gateways or provider APIs, model evaluation pipelines, coding agents in sandboxed environments, or agent memory systems and their databases.
• Experience serving models with vLLM or running distributed inference and training with Ray.
• Kubernetes controller or operator development, usage-based service cost management, or work in security-sensitive or regulated settings.
• Contributions to open-source gateway, proxy, or observability projects.