Staff + Senior Software Engineer, Inference Infrastructure

Menlo Ventures; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Anthropic is seeking a senior engineer to join the Inference team, building the systems that serve Claude to millions of users worldwide. You will work on distributed inference infrastructure, from request routing to orchestration across accelerators and clouds.

You will design resilient, high-performance components that adapt in real time to real-world events, enabling both reliable production workloads and ongoing research.

Qualifications

• Experience building large-scale distributed systems.
• Strong understanding of inference workloads and latency.
• Ability to design resilient, scalable software for production.

Responsibilities

• Design, build, and maintain distributed systems that serve Claude to millions of users worldwide.
• Develop resilient, flexible systems that adapt in real time to real world events.
• Develop intelligent request routing and fleet orchestration across accelerators and cloud platforms.

Skills

Distributed systems

Inference

Cloud deployment

Job description

About Anthropic

Anthropic's mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.

About the role

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators.

The team has a dual mandate: maximizing compute efficiency to reliably serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

Inference systems are highly performance sensitive distributed systems. Inference serves hundreds of thousands of customers every day, and the size & span of the inference fleet requires sophisticated routing, scaling, and networking systems.

Key responsibilities

• Design, build, and maintain the distributed systems that serve Claude to millions of users worldwide
• Develop resilient, flexible systems that adapt in real time to real world events
• Develop intelligent request routing,