Engineering Manager, ML Infrastructure & Fleet

Anthropic; New York (NY); Hybrid; USD 405,000 - 625,000; Full time

Benefits offered by this job

Competitive compensation

Equity donation

Vacation and parental leave

Flexible working hours

Office space NY

Job summary

Anthropic in New York seeks an experienced Senior Engineering Manager to lead the inference fleet control plane and ML platform teams. You will own the roadmap, drive capacity planning, and align across product, performance, and cloud teams to deliver reliable, scalable inference services.

You’ll hire and mentor engineers, shape the group structure, and ensure robust operations while advancing steerable, trustworthy AI initiatives.

Qualifications

• Engineering management experience leading teams on critical‑path production infrastructure at scale.
• A deep systems background — load balancing, scheduling, cluster orchestration, autoscaling, cache‑coherent distributed state, high‑performance networking, or similar.
• Experience shipping performance or efficiency improvements in large‑scale systems with quantifiable impact.
• Experience running production infrastructure with on‑call, incident response, capacity events, and deploy discipline.
• Results‑oriented, able to balance throughput, latency, cost, stability, and timelines while delivering impact.

Responsibilities

• Own the technical roadmap for the inference fleet coordination and capacity planning.
• Partner with product, inference engine, performance, and capacity teams to ship measurable improvements.
• Develop quantitative modeling to verify gains before shipping.
• Set technical strategy across heterogeneous hardware and multiple cloud providers.
• Run the group's operational backbone including on‑call rotations and incident response.
• Provide clarity at seams between API surface, inference engines, capacity planning, and deployment teams.
• Develop and retain engineers, and hire against a high technical bar.
• Coach engineers through a roadmap where priorities shift.

Skills

Engineering management

Systems engineering

Performance optimization

On-call

Cross-team collaboration

Education

Bachelor's degree

Tools

Kubernetes

Distributed systems

Job description

Anthropic in New York seeks an experienced Senior Engineering Manager to lead the inference fleet control plane and ML platform teams. You will own the roadmap, drive capacity planning, and align across product, performance, and cloud teams to deliver reliable, scalable inference services.

You’ll hire and mentor engineers, shape the group structure, and ensure robust operations while advancing steerable, trustworthy AI initiatives.