Staff Software Engineer, Scalable AI Inference Systems

Anthropic; San Francisco (CA); On-site; USD 320,000 - 485,000; Full time

Job summary

Anthropic seeks an experienced software engineer to design and scale distributed inference systems powering Claude. You will work on intelligent request routing, fleet orchestration, and multi-cloud deployment across accelerators to support millions of users.

You will collaborate with researchers and engineers to maximize compute efficiency, reduce costs, and deliver production-grade pipelines for model deployment while advancing ML infrastructure at scale.

Qualifications

• Significant software engineering experience, particularly with distributed systems.
• Bias toward flexibility, impact and ownership.
• Desire to learn about machine learning systems and infrastructure.
• Appreciates societal impacts of work.

Responsibilities

• Design, build, and maintain distributed systems powering Claude for millions of users.
• Develop resilient, real-time systems and intelligent request routing across cloud providers.
• Maximize compute efficiency and cost through autoscaling of production and research workloads.
• Create production-grade deployment pipelines for new models.
• Provide high-performance inference infrastructure for researchers.
• Integrate new accelerator platforms and model architectures.

Skills

Distributed systems

Results oriented

Adaptable

ML infra interest

Technical excellence

Societal impact

Research collaboration

Education

Bachelor’s degree in a relevant field

Tools

Kubernetes

Job description

Anthropic seeks an experienced software engineer to design and scale distributed inference systems powering Claude. You will work on intelligent request routing, fleet orchestration, and multi-cloud deployment across accelerators to support millions of users.

You will collaborate with researchers and engineers to maximize compute efficiency, reduce costs, and deliver production-grade pipelines for model deployment while advancing ML infrastructure at scale.