Senior AI Gateway Engineer - Remote, High-Impact

Docker; Oregon; Hybrid; USD 161,000 - 261,000; Full time

Benefits offered by this job

Remote-first culture

Flexibility in schedule

Generous PTO

Home-office support

Tech stipend

Learning & development

Parental leave

Equity

Comprehensive benefits

Docker swag

Job summary

Docker is seeking a Senior Backend Engineer to drive the AI Gateway platform. You will design core systems like the model router and MCP gateway, and build scalable infrastructure for dynamic routing and policy enforcement.

You'll own credential storage, session summarization, caching, and rate limiting, while delivering OpenAI-compatible APIs. This remote-first role offers global impact, equity, and strong benefits.

Qualifications

• 6+ years of backend engineering experience with production-grade systems.
• Deep knowledge of distributed and scalable systems, cloud-native infra, and API design.
• Experience building secure, high-throughput services (gateways, proxies, load balancers, policy engines).
• Fluency in Go, and/or Rust (both preferred).
• Familiarity with AI/ML platforms or model serving infra.
• Strong product mindset—building developer-facing tools.
• Bachelor's degree in CS, Engineering, or related field, or equivalent experience.

Responsibilities

• Design and implement core systems powering the AI Gateway, including the model router, MCP gateway, and control plane.
• Build infrastructure that supports dynamic model selection/routing, auto-failover, cost-based routing, and policy enforcement.
• Own secure credential storage, session summarization, caching, and rate limiting.
• Develop APIs for developers building with OpenAI-compatible interfaces and the Model Context Protocol.
• Build the underlying infrastructure to support evaluation, telemetry, replay, and backtesting for agents and LLM workflows.
• Lead architectural decisions and mentor engineers as the team scales.
• Performance tuning - caching, session summarization, and rate limiting that keep the gateway fast under production load.
• Contribute to roadmap planning, technical strategy, and cross-functional alignment.

Skills

Backend engineering

Distributed systems

Cloud-native infra

API design

Go

Rust

AI/ML platforms

Product mindset

Ownership mentality

Education

Bachelor's degree in Computer Science or Engineering

Tools

Kubernetes

Docker

OpenTelemetry

OPA

LangChain

Job description

Docker is seeking a Senior Backend Engineer to drive the AI Gateway platform. You will design core systems like the model router and MCP gateway, and build scalable infrastructure for dynamic routing and policy enforcement.

You'll own credential storage, session summarization, caching, and rate limiting, while delivering OpenAI-compatible APIs. This remote-first role offers global impact, equity, and strong benefits.