Staff Inference Runtime Architect (Rust/Python)

Anthropic; New York (NY); Hybrid; USD 405,000 - 485,000; Full time

Benefits offered by this job

Generous vacation

Parental leave

Flexible working hours

Equity donation matching

Job summary

Anthropic in New York is seeking a Staff Engineer to lead the Inference Runtime team, focusing on developing a high-performance, accelerator-agnostic runtime for inference services. In this senior role, you will drive technical direction, architectural design, and mentorship within the team.

The ideal candidate will have extensive experience in systems engineering, significant technical leadership, and the capability to optimize performance across multiple platforms, including GPU and TPU. Competitive salary and benefits offered.

Qualifications

• Deep background in systems engineering or ML infrastructure.
• Significant software engineering experience in large-scale distributed systems.
• Strong written and verbal communication skills.

Responsibilities

• Set technical direction for the inference serving stack.
• Own and evolve the accelerator-agnostic runtime.
• Drive efficient accelerator usage across multiple platforms.

Skills

Systems engineering

Performance profiling

Latency optimization

Throughput optimization

Communication

Education

Bachelor's degree in a relevant field

Tools

CUDA/GPU

TPU

Trainium

Kubernetes

Job description

Anthropic in New York is seeking a Staff Engineer to lead the Inference Runtime team, focusing on developing a high-performance, accelerator-agnostic runtime for inference services. In this senior role, you will drive technical direction, architectural design, and mentorship within the team.

The ideal candidate will have extensive experience in systems engineering, significant technical leadership, and the capability to optimize performance across multiple platforms, including GPU and TPU. Competitive salary and benefits offered.