Performance Engineer, Inference Systems

Anthropic Limited; San Francisco (CA); Hybrid; USD 350,000 - 850,000; Full time

Job summary

Anthropic seeks a Performance Engineer to optimize inference systems across throughput, latency, reliability, and correctness. You will run cross-layer investigations, improve evaluation pipelines, and build observability tools to surface metrics across the stack.

You will collaborate with kernel, serving, routing, and capacity teams to push high-impact optimizations, balancing speed with accuracy, and shaping the system for large-scale model deployment.

Qualifications

• Hands-on performance engineering experience in complex production systems.
• Strong Python programming and ability to contribute to large codebases you didn’t author.
• Solid data analysis skills (SQL, pandas) to turn telemetry into findings.

Responsibilities

• Run cross-layer performance investigations across throughput, latency, and reliability, identify root causes, and quantify gains.
• Own and improve the correctness evaluation pipeline that validates model output quality across hardware and configurations; lead investigations when regressions occur.
• Build observability dashboards and modeling tools that surface throughput, latency, cost, reliability, and their interactions across the stack.
• Partner with kernel, serving, routing, autoscaling, and capacity teams to implement high-impact optimizations.
• Ruthlessly rank opportunities by impact and effort, and decline lower-value items.

Skills

Profiling and roofline analysis

Latency & throughput optimization

Root-cause investigation

Python programming

Data analysis (SQL, pandas)

Effective written communication

Numerics and evaluation design

Education

Bachelor’s degree or equivalent

Job description

Anthropic seeks a Performance Engineer to optimize inference systems across throughput, latency, reliability, and correctness. You will run cross-layer investigations, improve evaluation pipelines, and build observability tools to surface metrics across the stack.

You will collaborate with kernel, serving, routing, and capacity teams to push high-impact optimizations, balancing speed with accuracy, and shaping the system for large-scale model deployment.