Senior AI Infrastructure Engineer, Rust & Scalable ML Ops

Wwshemi; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Kraken is seeking a senior engineer to design and operate the production infrastructure powering AI agent systems. You will develop high-performance Rust services for model inference, orchestration, and execution, building scalable systems that serve millions of users and high-throughput workloads.

You will drive ML infrastructure, MLOps patterns, observability, and failure recovery, while optimizing latency and cost across layers.

Qualifications

• 5+ years of experience building and operating high-scale production systems.
• Strong proficiency in Rust and systems-level programming.
• Deep understanding of distributed systems, reliability engineering, and performance optimization.
• Experience operating services serving millions of users or high-throughput workloads.
• Familiarity with ML infrastructure, model serving, or MLOps in production environments.
• Experience designing observability, monitoring, and failure recovery systems.
• Strong collaboration skills across infrastructure and applied engineering teams.
• High ownership mindset in high-stakes production environment.

Responsibilities

• Design and build the infrastructure layer powering AI agent systems in production
• Develop high-performance Rust services that handle model inference, orchestration, and execution
• Architect scalable systems capable of supporting millions of users and high request throughput
• Build reliable ML infrastructure and MLOps patterns for model deployment, evaluation, and monitoring
• Define guardrails, observability, and failure handling for agent-driven workflows
• Optimize latency, throughput, and cost across inference and orchestration layers
• Partner closely with the Agent Systems team to translate experimental prototypes into hardened production systems
• Contribute to foundational infrastructure decisions in a high-scale, high-impact environment

Skills

Rust

Distributed systems

Performance optimization

MLOps

Observability

Collaboration

Ownership

Tools

Kubernetes

Cloud-native infra

Job description

Kraken is seeking a senior engineer to design and operate the production infrastructure powering AI agent systems. You will develop high-performance Rust services for model inference, orchestration, and execution, building scalable systems that serve millions of users and high-throughput workloads.

You will drive ML infrastructure, MLOps patterns, observability, and failure recovery, while optimizing latency and cost across layers.