Staff & Sr. Software Engineer: Distributed Inference

Anthropic; New York (NY); Hybrid; USD 320,000 - 485,000; Full time

Benefits offered by this job

Competitive compensation

Benefits package

Equity donation matching (optional)

Generous vacation and parental leave

Flexible working hours

Office space and collaboration areas

Job summary

Anthropic is seeking a senior software engineer for its Inference team in New York. You will design and operate distributed systems powering Claude at scale, with intelligent routing, auto-scaling, and deployment pipelines across multiple cloud providers.

The role emphasizes compute efficiency and collaboration with researchers and engineers, requiring strong distributed systems background and experience with Python or Rust.

Qualifications

• Significant software engineering experience, particularly with distributed systems.
• Results-oriented with bias toward impact and flexibility.
• Willingness to take on extra duties outside the job description.
• Interest in learning more about machine learning systems and infrastructure.

Responsibilities

• Design, build, and maintain distributed systems that serve Claude to millions of users.
• Develop resilient, flexible systems that adapt in real time to events.
• Develop intelligent request routing, load balancing, and traffic management across accelerators and clouds.
• Maximize compute efficiency and optimize fleet cost via autoscaling and orchestration.
• Build and operate production-grade deployment pipelines for new models.
• Provide high-performance inference infrastructure for researchers.

Skills

Distributed systems

High-performance systems

Python

Rust

Team collaboration

Education

Bachelor's degree or equivalent

Tools

Kubernetes

AWS

GCP

Azure

Python

Rust

Job description

Anthropic is seeking a senior software engineer for its Inference team in New York. You will design and operate distributed systems powering Claude at scale, with intelligent routing, auto-scaling, and deployment pipelines across multiple cloud providers.

The role emphasizes compute efficiency and collaboration with researchers and engineers, requiring strong distributed systems background and experience with Python or Rust.