Engineering Manager, Inference Platform & Fleet

Anthropic Limited; New York (NY); Hybrid; USD 405,000 - 625,000; Full time

Job summary

Anthropic is hiring a senior engineering leader to own the inference fleet coordination and control-plane responsible for routing traffic, placing capacity, and ensuring fast, reliable inference across clouds. You will lead ML platform and distributed-systems engineers, shaping architecture and driving performance improvements at scale.

You will partner with product, inference engines, and capacity teams to optimize throughput, latency, and cost, while managing on-call rotations and deployments

Qualifications

• Engineering management experience leading teams on critical-path production infrastructure at scale.
• A deep systems background — load balancing, scheduling, cluster orchestration, autoscaling, cache-coherent distributed state, high-performance networking.
• Experience shipping performance or efficiency improvements in large-scale systems with measurable impact (cost and latency).
• Experience running production infrastructure with on-call, incident response, capacity events, and deploy discipline.
• Results-oriented, able to balance throughput, latency, cost, stability, and timelines while delivering impact.
• Ability to build strong relationships across teams; this is a seam role with cross-team dependencies.
• Curiosity about machine learning systems and transformer inference how it shapes systems.

Responsibilities

• Own the technical roadmap for the inference fleet coordination: traffic routing, capacity placement, cache strategy, and control-plane protocols.
• Collaborate with product, inference engine, performance, and capacity teams to find throughput, latency, utilization, and cost wins and ship improvements.
• Build a habit of quantitative modeling; measure wins and predict effects before shipping.
• Set technical strategy for cross-hardware, multi-cloud, and all serving surfaces.
• Oversee on-call rotations, incident response, postmortems, and deployment discipline to keep the system robust.
• Clarify interfaces between API surface, inference engines, capacity planning, and cloud deployment teams.
• Develop and retain strong teams, and recruit to raise the technical bar.

Skills

Engineering management

Systems depth

Performance optimization

On-call / incident response

Cross-team collaboration

ML infrastructure familiarity

Transformer inference familiarity

Education

Bachelor’s degree or equivalent

Tools

Kubernetes internals

Borg-style systems

Cloud providers

Job description

Anthropic is hiring a senior engineering leader to own the inference fleet coordination and control-plane responsible for routing traffic, placing capacity, and ensuring fast, reliable inference across clouds. You will lead ML platform and distributed-systems engineers, shaping architecture and driving performance improvements at scale.

You will partner with product, inference engines, and capacity teams to optimize throughput, latency, and cost, while managing on-call rotations and deployments