Staff Software Engineer, Inference Systems at Scale

Anthropic; San Francisco (CA); On-site; USD 300,000 - 485,000; Full time

Benefits offered by this job

Competitive salary

Flexible working hours

Generous vacation and parental leave

Job summary

A forward-thinking AI company in San Francisco is seeking an experienced software engineer to join their Inference team. The role focuses on building and optimizing systems for AI model deployment, with a strong emphasis on technical excellence and societal impact. Candidates should possess solid software engineering experience, particularly in distributed systems, and be eager to work in a collaborative environment. Competitive salary range of $300,000 to $485,000 USD and hybrid work policy are offered.

Qualifications

• Experience with large-scale distributed systems is a plus.
• Experience with machine learning systems at scale is a plus.
• Experience with load balancing and traffic management is a plus.
• Optimizing LLM inference and caching strategies is advantageous.

Responsibilities

• Building and maintaining inference systems for AI models.
• Designing routing algorithms for request distribution.
• Autoscaling compute fleet according to demand.
• Managing multi-region deployments and analysis of observability data.

Skills

Software engineering experience

Distributed systems

Flexibility

Impact-oriented

Pair programming

Understanding of machine learning systems

Technical excellence

Concern for societal impacts

Education

Bachelor's degree in a related field

Tools

Kubernetes

Cloud infrastructure (AWS, GCP)

Python

Rust

Job description

A forward-thinking AI company in San Francisco is seeking an experienced software engineer to join their Inference team. The role focuses on building and optimizing systems for AI model deployment, with a strong emphasis on technical excellence and societal impact. Candidates should possess solid software engineering experience, particularly in distributed systems, and be eager to work in a collaborative environment. Competitive salary range of $300,000 to $485,000 USD and hybrid work policy are offered.