Performance Engineer, Inference Systems

Anthropic Limited; Seattle (WA); Hybrid; USD 350,000 - 850,000; Full time

Job summary

Anthropic is seeking a Performance Engineer to optimize inference fleet throughput, latency, and reliability across hardware and software layers. You’ll validate model outputs for correctness and work with kernel, routing, and autoscaling teams to drive high-impact optimizations.

You’ll build observability and modeling tools to make system behavior observable and actionable, prioritizing improvements by impact while maintaining a strong focus on numerical rigor and regression detection.

Qualifications

• Profiling, roofline analysis, latency/throughput optimization, and root-cause investigation in complex production systems.
• Ability to read, instrument, and contribute to large production codebases you didn’t write
• Strong data analysis skills with telemetry to findings
• Ability to communicate quantitative results in writing to influence priorities
• Genuine interest in correctness as an engineering discipline: numerics, evaluation design, regression detection

Responsibilities

• Run cross-layer performance investigations across throughput, latency, and reliability, sizing the gap between actual fleet performance and theoretical rooflines
• Own and improve the correctness evaluation pipeline that validates model output quality across hardware platforms, numerics, and serving configurations
• Build observability, dashboards, and modeling tools that make throughput, latency, cost, reliability, and their interactions legible across the stack
• Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize optimizations
• Ruthlessly stack-rank opportunities by impact and effort, saying no to non-critical items

Skills

Performance engineering

Python

Data analysis

Communication

Correctness discipline

Education

Bachelor’s degree

Tools

SQL

Pandas

Job description

Anthropic is seeking a Performance Engineer to optimize inference fleet throughput, latency, and reliability across hardware and software layers. You’ll validate model outputs for correctness and work with kernel, routing, and autoscaling teams to drive high-impact optimizations.

You’ll build observability and modeling tools to make system behavior observable and actionable, prioritizing improvements by impact while maintaining a strong focus on numerical rigor and regression detection.