Engineering Manager, AI Inference Platform — Flexible Hours

Neura Market; San Francisco, Northern (CA, KY); Hybrid; USD 405,000 - 625,000; Full time

Job summary

Anthropic, based in San Francisco, is seeking an engineering leader to own the technical roadmap for coordinating the inference fleet and to guide a diverse group of ML platform, infrastructure, and distributed-systems engineers. You’ll shape architecture, drive reliability, and ensure the end-to-end path from request to model stays efficient as models and clouds evolve.

You will partner with product, inference engine, performance, and capacity teams to push throughput, latency, and cost

Qualifications

• Engineering management experience leading critical-path production infrastructure at scale.
• Deep systems background—load balancing, scheduling, autoscaling, distributed state.
• Experience shipping performance improvements with measurable impact, including cost.
• Experience running production infrastructure with on-call, incident response.
• Results-oriented with balance of throughput, latency, cost, and delivery timelines.
• Ability to build strong relationships across team boundaries ( seam role ).
• Curiosity about machine learning systems and transformer inference.

Responsibilities

• Own the technical roadmap for how the inference fleet is coordinated: traffic routing, capacity placement, and synchronization protocols.
• Partner with product, inference engine, performance, and capacity teams to identify throughput, latency, utilization, and cost wins and ship improvements.
• Build the group’s habit of quantitative modeling and measure impact before shipping.
• Set technical strategy for control plane evolution across heterogeneous hardware and multiple clouds.
• Run the group’s on-call rotations, incident response, deploy safety, and ensure stability while shipping aggressively.
• Create clarity at the seam between API surface, inference engines, capacity planning, and cloud deployment.

Skills

Engineering management

Systems depth

Performance optimization

Production infrastructure

Cross-team leadership

ML systems curiosity

Education

Bachelor's degree

Tools

Kubernetes

Job description

Anthropic, based in San Francisco, is seeking an engineering leader to own the technical roadmap for coordinating the inference fleet and to guide a diverse group of ML platform, infrastructure, and distributed-systems engineers. You’ll shape architecture, drive reliability, and ensure the end-to-end path from request to model stays efficient as models and clouds evolve.

You will partner with product, inference engine, performance, and capacity teams to push throughput, latency, and cost