Inference Systems Performance Engineer

Anthropic Limited; New York (NY); Hybrid; USD 350,000 - 850,000; Full time

Job summary

Anthropic is seeking a Performance Engineer to analyze cross-layer bottlenecks in inference systems and drive improvements in throughput, latency, and correctness. You will own evaluation pipelines, build observability dashboards, and partner with kernel, serving, routing, and autoscaling teams to land impactful optimizations.

The role emphasizes rigorous data analysis, Python instrumentation, and clear communication of results to influence decisions across complex production systems.

Qualifications

• Hands-on performance engineering experience: profiling, roofline analysis, latency/throughput optimization, and root‑cause investigations in complex production systems.
• Proficiency in Python with the ability to read, instrument, and contribute to large production codebases you didn’t write.
• Solid data analysis skills (e.g. SQL, pandas, or similar) to turn telemetry into clear findings.
• Ability to communicate quantitative results clearly in writing to influence priorities on teams you don’t manage.
• Genuine interest in correctness as an engineering discipline: numerics, evaluation design, regression detection.

Responsibilities

• Run cross-layer performance investigations across throughput, latency, and reliability; quantify the gap between actual fleet performance and theoretical rooflines.
• Own and improve the correctness evaluation pipeline that validates model output quality across hardware platforms, numerics, and serving configurations.
• Build observability, dashboards, and modeling tools that make throughput, latency, cost, reliability, and correctness visible across the stack.
• Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize and land high-impact optimizations found in analyses.
• Ruthlessly stack-rank opportunities by impact and effort, and say no to non-value-added tasks.

Skills

Performance engineering

Python

SQL / Data analysis

Effective communication

Numerics / evaluation design

Education

Bachelor’s degree

Job description

Anthropic is seeking a Performance Engineer to analyze cross-layer bottlenecks in inference systems and drive improvements in throughput, latency, and correctness. You will own evaluation pipelines, build observability dashboards, and partner with kernel, serving, routing, and autoscaling teams to land impactful optimizations.

The role emphasizes rigorous data analysis, Python instrumentation, and clear communication of results to influence decisions across complex production systems.