Staff Software Engineer, Scalable Inference Platforms

Anthropic Limited; New York (NY); Hybrid; USD 320,000 - 485,000; Full time

Benefits offered by this job

Equity donation matching

Generous vacation and parental leave

Flexible working hours

Office in San Francisco

Job summary

Anthropic is seeking an experienced engineer to join the Inference team, delivering the Claude inference stack across global deployments. You will design scalable, high-performance distributed systems and drive real-time routing, load balancing, and efficient compute orchestration.

You will collaborate closely with researchers and engineers to deploy new models, optimize inference performance, and ensure reliability across multi-cloud environments while supporting production, research, and

Qualifications

• Extensive software engineering experience with distributed systems.
• Strong ownership mindset with impact-focused delivery.
• Willing to take on tasks beyond the formal role scope.
• Interest in ML systems and infrastructure.

Responsibilities

• Design and maintain the distributed inference systems behind Claude.
• Develop resilient, real-time routing, load balancing, and traffic management.
• Scale compute and orchestrate production, research, and experimental workloads.
• Build deployment pipelines for model releases and observability.

Skills

Distributed systems

High performance

Load balancing

Request routing

Fault tolerance

Python

Rust

Cloud platforms

Education

Bachelor's degree

Tools

Kubernetes

AWS

GCP

Azure

CI/CD pipelines

Job description

Anthropic is seeking an experienced engineer to join the Inference team, delivering the Claude inference stack across global deployments. You will design scalable, high-performance distributed systems and drive real-time routing, load balancing, and efficient compute orchestration.

You will collaborate closely with researchers and engineers to deploy new models, optimize inference performance, and ensure reliability across multi-cloud environments while supporting production, research, and