Senior Staff Engineer, Distributed Inference Systems

Anthropic Limited; New York (NY); Hybrid; USD 320,000 - 485,000; Full time

Job summary

Anthropic is seeking a senior engineer to scale Claude’s inference infrastructure. You will design and operate distributed systems across multiple cloud providers and accelerators, ensuring high performance, reliability, and cost efficiency.

You will collaborate with researchers and engineers to deploy new models and optimize routing and scheduling at scale. The role emphasizes building production-grade deployment pipelines, integrating new accelerator platforms, and maintaining robust inference

Qualifications

• Bachelor’s degree or equivalent in a relevant field.
• Experience with high-performance distributed systems.
• Experience with Kubernetes and cloud infrastructure.
• Proficiency in Python or Rust.

Responsibilities

• Design, build, and maintain distributed systems that serve Claude to millions of users worldwide.
• Develop resilient, flexible systems that adapt in real time to real world events.
• Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators and multiple cloud providers.
• Maximize compute efficiency and optimize cost across the fleet by autoscaling and orchestrating production, research, and experimental workloads across multiple cloud providers.
• Build and operate production-grade deployment pipelines for releasing new models to users.
• Provide high-performance inference infrastructure that enables researchers to develop next-generation models.
• Integrate new AI accelerator platforms and support inference for new model architectures.

Skills

Distributed systems

Load balancing

Request routing

Traffic management

Kubernetes

Cloud infrastructure

Python

Rust

Education

Bachelor’s degree

Tools

AWS

GCP

Azure

Job description

Anthropic is seeking a senior engineer to scale Claude’s inference infrastructure. You will design and operate distributed systems across multiple cloud providers and accelerators, ensuring high performance, reliability, and cost efficiency.

You will collaborate with researchers and engineers to deploy new models and optimize routing and scheduling at scale. The role emphasizes building production-grade deployment pipelines, integrating new accelerator platforms, and maintaining robust inference