Senior Software Engineer - Dev Platform

kraken.com; United States; Remote; USD 120,000 - 200,000; Full time

Job summary

Kraken is seeking an experienced backend platform engineer to build and operate an internal AI gateway that gives engineering teams governed access to large language model providers. The platform supports model selection, access controls, usage visibility, and safety safeguards, helping teams adopt LLM capabilities at scale.

You’ll own features from design through production, collaborate with security and reliability partners, and help migrate workloads while delivering well-documented APIs and

Qualifications

• Production experience building and operating backend APIs or distributed services.
• Proficiency in a backend language such as Rust, Python, or TypeScript.
• Experience with LLM APIs, including authentication, streaming, token accounting, rate limits, and provider failures.
• Familiar with observability, CI/CD, containers; Kubernetes and major cloud provider.
• Ability to design clear, documented APIs for internal users.
• Strong communication and ownership from proposal through deployment.

Responsibilities

• Add model and provider integrations, routing rules, budgets, rate limits, and cost reporting.
• Design APIs, SDKs, and self-service tools for onboarding internal engineering teams.
• Help migrate existing LLM workloads with minimal disruption.
• Implement access control, key management, and audit logging with security specialists.
• Instrument services to expose latency, errors, token usage, and costs for each workload.
• Own production features, support internal users with documentation and debugging, and participate in reviews and design practices.

Skills

Backend APIs

Distributed systems

Rust

Python/TypeScript

CI/CD

Observability

Kubernetes

Cloud platforms

API design

Documentation

Ownership

Communication

Tools

Kubernetes

Cloud platforms

CI/CD tools

API frameworks

Job description

Role overview

Build and operate an internal AI gateway that gives engineering teams governed access to large language model providers. The platform supports model selection, access controls, usage visibility, and operational safeguards, helping teams adopt LLM capabilities while managing cost, reliability, and security. You’ll own features from design through production and work closely with platform, security, and reliability partners.

Responsibilities

• Add model and provider integrations, routing rules, budgets, rate limits, and usage and cost reporting.
• Design APIs, SDKs, and self-service tools that make onboarding straightforward for internal engineering teams.
• Help migrate existing LLM workloads with minimal disruption.
• Implement access control, key management, and audit logging in partnership with security specialists.
• Instrument services to expose latency, errors, token usage, and costs for each workload.
• Own production features, support internal users with documentation and debugging, and contribute to reviews and design practices.

Requirements

• Production experience building and operating backend APIs or distributed services.
• Strong ability in a backend language such as Rust, Python, or TypeScript, with a focus on maintainable, tested code.
• Working knowledge of LLM APIs, including authentication, streaming, token accounting, rate limits, and provider failures.
• Experience with observability, CI/CD, and containers, plus familiarity with Kubernetes and a major cloud provider.
• Ability to design clear, documented APIs for internal users and incorporate their feedback.
• Strong communication and ownership, carrying work from proposal through deployment.

Nice to have

• Experience with API gateways, proxies, or traffic management.
• Experience building LLM systems involving routing, caching, evaluation, or cost controls.
• Familiarity with OAuth, OIDC, or role-based access control.
• Contributions to open-source infrastructure or developer platform projects.