Senior AI Platform Engineer - LLM Gateway & Governance

kraken.com; United States; Remote; USD 120,000 - 200,000; Full time

Job summary

Kraken is seeking an experienced backend platform engineer to build and operate an internal AI gateway that gives engineering teams governed access to large language model providers. The platform supports model selection, access controls, usage visibility, and safety safeguards, helping teams adopt LLM capabilities at scale.

You’ll own features from design through production, collaborate with security and reliability partners, and help migrate workloads while delivering well-documented APIs and

Qualifications

• Production experience building and operating backend APIs or distributed services.
• Proficiency in a backend language such as Rust, Python, or TypeScript.
• Experience with LLM APIs, including authentication, streaming, token accounting, rate limits, and provider failures.
• Familiar with observability, CI/CD, containers; Kubernetes and major cloud provider.
• Ability to design clear, documented APIs for internal users.
• Strong communication and ownership from proposal through deployment.

Responsibilities

• Add model and provider integrations, routing rules, budgets, rate limits, and cost reporting.
• Design APIs, SDKs, and self-service tools for onboarding internal engineering teams.
• Help migrate existing LLM workloads with minimal disruption.
• Implement access control, key management, and audit logging with security specialists.
• Instrument services to expose latency, errors, token usage, and costs for each workload.
• Own production features, support internal users with documentation and debugging, and participate in reviews and design practices.

Skills

Backend APIs

Distributed systems

Rust

Python/TypeScript

CI/CD

Observability

Kubernetes

Cloud platforms

API design

Documentation

Ownership

Communication

Tools

Kubernetes

Cloud platforms

CI/CD tools

API frameworks

Job description

Kraken is seeking an experienced backend platform engineer to build and operate an internal AI gateway that gives engineering teams governed access to large language model providers. The platform supports model selection, access controls, usage visibility, and safety safeguards, helping teams adopt LLM capabilities at scale.

You’ll own features from design through production, collaborate with security and reliability partners, and help migrate workloads while delivering well-documented APIs and