Staff Software Engineer, AI Inference Systems

Anthropic; New York (NY); Hybrid; USD 320,000 - 485,000; Full time

Benefits offered by this job

Competitive compensation

Equity donation matching (optional)

Generous vacation

Parental leave

Flexible working hours

Office space

Job summary

Anthropic is seeking an experienced engineer for the Inference team to design, build, and scale distributed systems serving Claude to millions of users. You will develop intelligent request routing, load balancing, and orchestration across cloud providers and accelerators, while maximizing compute efficiency and enabling researchers.

The role emphasizes production-grade pipelines, multi-region deployments, and high-performance infrastructure.

Qualifications

• Significant software engineering experience, particularly with distributed systems.
• Results-oriented, with a bias towards flexibility and impact.
• Willingness to pick up slack, even if it goes outside your job description.
• Desire to learn more about machine learning systems and infrastructure.
• Thrive in environments where technical excellence directly drives both business results and research breakthroughs.
• Care about the societal impacts of your work.

Responsibilities

• Design, build, and maintain the distributed systems that serve Claude to millions of users worldwide.
• Develop resilient, flexible systems that adapt in real time to real world events.
• Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators and multiple cloud providers.
• Maximize compute efficiency and optimize cost across the fleet by autoscaling and orchestrating production, research, and experimental workloads across multiple cloud providers.
• Build and operate production-grade deployment pipelines for releasing new models to users.
• Provide high-performance inference infrastructure that enables researchers to develop next-generation models.
• Integrate new AI accelerator platforms and support inference for new model architectures.

Skills

Distributed systems

Results-driven

Adaptable

ML systems awareness

Societal impact

Education

Bachelor’s degree in a relevant field

Tools

Kubernetes

AWS

GCP

Azure

Python

Rust

Job description

Anthropic is seeking an experienced engineer for the Inference team to design, build, and scale distributed systems serving Claude to millions of users. You will develop intelligent request routing, load balancing, and orchestration across cloud providers and accelerators, while maximizing compute efficiency and enabling researchers.

The role emphasizes production-grade pipelines, multi-region deployments, and high-performance infrastructure.