Senior Cloud Inference Engineer - Scalable LLM Serving

EngineersOfAI; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Anthropic in San Francisco is seeking a Cloud Inference Engineer to scale and optimize Claude across AWS, GCP, and Azure. You will own validation, deployment, and performance of inference servers and load balancers on multiple CSPs, ensuring correctness and reliability.

You will design CI/CD for inference systems, reduce cycle times, and collaborate with cross-functional teams to push frontier models into production.

Qualifications

• Strong interest in LLM serving; prior inference or ML experience not required
• Extensive software engineering experience with high-performance, distributed systems
• Experience building or operating services on at least one major cloud platform
• Ability to collaborate across internal teams and external partners

Responsibilities

• Be on the critical path for frontier model launches and cloud platform integration
• Collaborate with core inference teams to bring new features to production
• Identify and fix cross-platform gaps in observability and deployment patterns
• Design, build, and own CI/CD for the inference server and load balancer across clouds
• Reduce validation cycle time while maintaining reliability and safety
• Analyze observability data to drive performance and cost optimizations

Skills

LLM serving

Distributed systems

Automation / test infra

Cross-functional collaboration

Cloud platforms (AWS, GCP, Azure)

Python / Rust

Tools

Kubernetes

Infrastructure as Code

Job description

Anthropic in San Francisco is seeking a Cloud Inference Engineer to scale and optimize Claude across AWS, GCP, and Azure. You will own validation, deployment, and performance of inference servers and load balancers on multiple CSPs, ensuring correctness and reliability.

You will design CI/CD for inference systems, reduce cycle times, and collaborate with cross-functional teams to push frontier models into production.