Senior AI Gateway Engineer – Remote, High-Impact Systems

Bazeta; United States; Remote; USD 161,000 - 260,000; Full time

Job summary

Docker is seeking a Senior Software Engineer for the AI Gateway in the United States. You will design and implement core systems powering the gateway, enabling secure, scalable routing and policy enforcement for AI agents.

You will own critical capabilities such as credential storage, session summarization, caching, and rate limiting, while collaborating on APIs and model context protocols. You will mentor engineers, influence architecture, and drive performance optimizations to keep latency low

Qualifications

• 6+ years of backend engineering experience with production-grade systems.
• Distributed and highly scalable systems, cloud-native infra, and API design.
• Secure, high-throughput services (gateways, proxies, load balancers, policy engines).
• Fluency in Go and/or Rust (both preferred).
• Familiarity with AI/ML platforms or model serving infra.
• Bachelor’s degree in CS/Engineering or equivalent.

Responsibilities

• Design and implement core systems powering the AI Gateway, including the model router, MCP gateway, and control plane.
• Build infrastructure for dynamic model selection/routing, auto-failover, cost-based routing, and policy enforcement.
• Own credential storage, session summarization, caching, and rate limiting.
• Develop APIs for OpenAI-compatible interfaces and the Model Context Protocol.
• Support evaluation, telemetry, replay, and backtesting for agents and LLM workflows.
• Lead architectural decisions and mentor engineers as the team scales.
• Performance tuning: caching, session summarization, and rate limiting for production load.
• Contribute to roadmap planning and cross-functional alignment.

Skills

Backend experience

Distributed systems

Cloud-native infra

APIs design

Go

Rust

AI/ML platforms

Product mindset

Education

Bachelor's degree in CS/Engineering

Tools

Kubernetes

Docker

OpenTelemetry

Job description

Docker is seeking a Senior Software Engineer for the AI Gateway in the United States. You will design and implement core systems powering the gateway, enabling secure, scalable routing and policy enforcement for AI agents.

You will own critical capabilities such as credential storage, session summarization, caching, and rate limiting, while collaborating on APIs and model context protocols. You will mentor engineers, influence architecture, and drive performance optimizations to keep latency low