Engineering Manager, ML Efficiency - AI Rapid Response Lead

Google; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google in Mountain View seeks an Engineering Manager for the ML Efficiency team. You will guide a team of AI/ML systems engineers, balance technical leadership with people management, and steer architecture for large-scale pipelines.

You will lead sprints, shape strategy, and collaborate with cross-functional teams across Google Cloud to deliver high-impact ML solutions and TVPs in a fast-paced environment.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of software development experience.
• 5 years testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with ML domains: speech/audio, reinforcement learning, ML infrastructure, or related field.
• 5 years of experience with ML design/infrastructure: deployment, evaluation, data processing, debugging, fine tuning.
• Experience integrating generative AI tools or LLM interfaces into workflows.

Responsibilities

• Lead technical pathfinding and system design for the ML Efficiency Hub, driving complex 1–6 month Engineers and 2–4 week Strike Sprints.
• Design, prototype, and write production C++ and Python code for model distillation, speculative decoding, dynamic batching, and distributed serving systems.
• Translate executive mandates into de-risked, latency-aware TVPs and concrete architectural decisions.
• Perform deep compute surgery on legacy pipelines, evaluate architectural trade-offs, and establish Graceful Exit Packages for autonomous operation.

Skills

Software development

ML design

Generative AI integration

C++

Python

System architecture

Leadership

Education

Bachelor's degree or equivalent practical experience

Tools

C++

Python

SavedModel

Pathways

BrainServer

XManager

Job description

Google in Mountain View seeks an Engineering Manager for the ML Efficiency team. You will guide a team of AI/ML systems engineers, balance technical leadership with people management, and steer architecture for large-scale pipelines.

You will lead sprints, shape strategy, and collaborate with cross-functional teams across Google Cloud to deliver high-impact ML solutions and TVPs in a fast-paced environment.