Engineering Manager: Inference Infrastructure Leader

EngineersOfAI; New York, Northern (NY, KY); Hybrid; USD 230,000 - 360,000; Full time

Job summary

Anthropic is seeking an engineering leader to head a group of ML platform, infrastructure, and distributed-systems engineers. You will own the technical roadmap for the inference fleet, ensure throughput and reliability, and guide architectural decisions that affect the entire path from request to model.

You will hire, mentor, and shape a team that can operate across hardware, clouds, and serving surfaces. The ideal candidate has a deep systems background, proven leadership of critical-path

Qualifications

• Engineering management experience leading teams on critical-path production infrastructure at scale.
• Deep systems background with depth to make architectural calls on a large fleet.
• Experience shipping performance or efficiency improvements with measurable impact.
• Experience running production infrastructure with on-call, incident response, and deployment discipline.
• Results-oriented mindset balancing throughput, latency, cost, and launch timelines.

Responsibilities

• Own the technical roadmap for inference fleet coordination, capacity, and cache placement.
• Collaborate with product, inference engine, and capacity teams to ship measurable throughput and latency improvements.
• Promote quantitative modeling; require measurement of wins before shipping.
• Set technical strategy for control plane across heterogeneous hardware and multi-cloud environments.
• Run the team’s operational backbone: on-call rotations, incident response, and postmortems.

Skills

Engineering management

Systems background

Load balancing

Capacity planning

Performance improvements

Tools

Cluster orchestration

Caching systems

Job description

Anthropic is seeking an engineering leader to head a group of ML platform, infrastructure, and distributed-systems engineers. You will own the technical roadmap for the inference fleet, ensure throughput and reliability, and guide architectural decisions that affect the entire path from request to model.

You will hire, mentor, and shape a team that can operate across hardware, clouds, and serving surfaces. The ideal candidate has a deep systems background, proven leadership of critical-path