Performance Engineer, Inference Engine - High-Performance AI

EngineersOfAI; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Anthropic is seeking a Performance Engineer for its Inference Engine to optimize LLM processing across accelerator and cloud platforms. You will work on throughput, latency, and cost, ensuring robust, scalable performance from host to device and across chips.

Experience with Rust/C++ and GPU/accelerator programming is essential. The role emphasizes strong systems thinking, observability, and collaboration in a fast-moving AI safety-focused environment.

Qualifications

• LLM inference: prefill and decode on accelerator compute.
• Proven quick learner: ramped fast in deep, unfamiliar systems.
• Strong systems programming (Rust, C++, or similar).
• Analytical about performance: observe, hypothesize, test, then measure.
• Low ego: receptive to feedback and collaboration.
• Enjoy pair programming and consider societal impact of work.

Responsibilities

• Keep device utilization high; accelerators should not wait due to overhead.
• Reuse state to avoid recompute when cheaper than recomputing.
• Build observability to model impact of potential improvements.
• Deploy improvements across hosts and devices while maintaining safety.

Skills

LLM inference

Rust

C++

systems programming

performance analysis

pair programming

analytical mindset

memory optimization

GPU/Accelerator programming

Tools

CUDA

RDMA

PCIe

Job description

Anthropic is seeking a Performance Engineer for its Inference Engine to optimize LLM processing across accelerator and cloud platforms. You will work on throughput, latency, and cost, ensuring robust, scalable performance from host to device and across chips.

Experience with Rust/C++ and GPU/accelerator programming is essential. The role emphasizes strong systems thinking, observability, and collaboration in a fast-moving AI safety-focused environment.