Staff Inference Runtime Architect (Rust/Python)

Anthropic; San Francisco (CA); Hybrid; USD 405,000 - 485,000; Full time

Benefits offered by this job

Competitive compensation

Flexible working hours

Generous vacation and parental leave

Job summary

Anthropic is seeking a Staff Engineer to lead the technical direction for the Inference Runtime. This senior role will involve optimizing a shared runtime across accelerators while mentoring fellow engineers to enhance technical standards.

Ideal candidates should have extensive experience in systems engineering, ML infrastructure, and possess software engineering expertise with distributed systems. A Bachelor’s degree is required, along with the ability to collaborate across technical and organizational boundaries.

Qualifications

• Deep background in systems engineering or ML infrastructure.
• Experience with accelerator ecosystems like CUDA/GPU.
• Significant software engineering experience.

Responsibilities

• Set technical direction for the inference serving stack.
• Own and evolve the accelerator-agnostic runtime.
• Drive efficient accelerator usage—including GPU and TPU.

Skills

Systems engineering

Performance profiling

Latency optimization

High-performance distributed systems

Bachelor's degree

Education

Bachelor's degree or equivalent

Tools

CUDA

Kubernetes

Job description

Anthropic is seeking a Staff Engineer to lead the technical direction for the Inference Runtime. This senior role will involve optimizing a shared runtime across accelerators while mentoring fellow engineers to enhance technical standards.

Ideal candidates should have extensive experience in systems engineering, ML infrastructure, and possess software engineering expertise with distributed systems. A Bachelor’s degree is required, along with the ability to collaborate across technical and organizational boundaries.