Staff Software Engineer - Large-Scale Inference

Fdetimes; Greater London; Hybrid; GBP 325,000 - 390,000; Full time

Job summary

Anthropic is building scalable inference systems to serve Claude to millions of users worldwide. The Inference team owns the end-to-end stack, from routing requests to fleet orchestration across diverse accelerators and cloud platforms.

As a member, you design resilient distributed systems, optimize compute for real-time events, and deploy production pipelines for new models while enabling researchers to push the frontier of AI safety and usefulness.

Qualifications

• Proficiency in Python or Rust.
• Experience building and operating distributed systems in production.
• Knowledge of containerized infrastructure (Kubernetes) and at least one major cloud platform.
• Results-oriented, with a bias toward flexibility and impact.
• Willingness to pick up slack, even if outside your job description.
• Desire to learn more about machine learning systems and infrastructure.
• Care about societal impacts of your work.

Responsibilities

• Design, build, and maintain the distributed systems that serve Claude to millions of users worldwide.
• Develop resilient, flexible systems that adapt in real time to real-world events.
• Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators.
• Maximize compute efficiency across the fleet by autoscaling and orchestrating production, research, and experimental workloads.
• Build and operate production-grade deployment pipelines for releasing new models to users.
• Provide high-performance inference infrastructure that enables researchers to develop next-generation models.
• Integrate new AI accelerator platforms and support inference for new model architectures.

Skills

Python

Rust

Distributed systems

Kubernetes

AWS

GCP

Azure

Tools

Kubernetes

AWS

GCP

Azure

Job description

Anthropic is building scalable inference systems to serve Claude to millions of users worldwide. The Inference team owns the end-to-end stack, from routing requests to fleet orchestration across diverse accelerators and cloud platforms.

As a member, you design resilient distributed systems, optimize compute for real-time events, and deploy production pipelines for new models while enabling researchers to push the frontier of AI safety and usefulness.