Staff Engineer — Inference Runtime Lead

Anthropic; New York (NY); Hybrid; USD 405,000 - 485,000; Full time

Job summary

Anthropic is looking for a Staff Software Engineer to lead the inference runtime. This pivotal role involves setting technical direction and evolving the architecture across accelerators. You will drive work that enhances performance, usability, and correctness while mentoring a team of engineers.

With over 8 years of experience in software engineering, you will manage technical strategies and contribute significantly to organizational collaborations. The position provides a hybrid work environment, emphasizing technology innovation in AI systems.

Qualifications

• 8+ years of software engineering experience, especially as a technical lead.
• Background with ML compiler toolchains or accelerator management.
• Experience with CI/CD systems for hardware workloads.

Responsibilities

• Set technical direction for the inference serving runtime architecture.
• Own and evolve the accelerator-agnostic runtime for performance and correctness.
• Mentor engineers to raise the technical bar and improve systems.

Skills

Systems engineering expertise

Experience with ML infrastructure

Proficiency in Rust and Python

Knowledge of CUDA/GPU, TPU, or Trainium

Strong communication skills

Education

Bachelor’s degree or equivalent experience

Tools

Kubernetes

ML compiler toolchains

Job description

Anthropic is looking for a Staff Software Engineer to lead the inference runtime. This pivotal role involves setting technical direction and evolving the architecture across accelerators. You will drive work that enhances performance, usability, and correctness while mentoring a team of engineers.

With over 8 years of experience in software engineering, you will manage technical strategies and contribute significantly to organizational collaborations. The position provides a hybrid work environment, emphasizing technology innovation in AI systems.