Engineering Manager, ML Efficiency & AI Systems

Socket.dev; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google Cloud seeks an Engineering Manager for the ML Efficiency team to drive technical architecture and engineering leadership across AI/ML systems. You will oversee multiple teams, budgets, and large-scale deployments across sites, guiding both strategy and execution.

You will balance hands-on coding in C++ and Python with organizational leadership, shaping roadmaps, mentoring engineers, and delivering scalable ML pipelines in a fast-paced environment.

Qualifications

• Bachelor’s degree or equivalent practical experience required.
• 8 years of experience in software development.
• 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with one or more ML domains (Speech/audio, reinforcement learning, ML infrastructure, or related fields).
• 5 years of experience with ML design and ML infrastructure (model deployment, evaluation, data processing, debugging, fine tuning).
• Experience integrating generative AI tools or LLM interfaces into workflows.

Responsibilities

• Lead technical pathfinding and system design for the ML Efficiency Hub, driving multi-month projects and sprint cycles.
• Design, prototype, and write production C++ and Python code for model distillation and distributed serving systems.
• Translate executive mandates into feasible technical plans while meeting latency and resource constraints.
• Evaluate architectural trade-offs and establish practical handoff strategies to enable future autonomy.

Skills

Software development

Software testing

Software architecture

ML/AI experience

Generative AI integration

Speech/audio ML

Reinforcement learning

Education

Bachelor’s degree or equivalent practical experience

Master’s degree or PhD in Engineering / CS / related field

Tools

C++

Python

Google infra (SavedModel/Pathways)

ML deployment

Job description

Google Cloud seeks an Engineering Manager for the ML Efficiency team to drive technical architecture and engineering leadership across AI/ML systems. You will oversee multiple teams, budgets, and large-scale deployments across sites, guiding both strategy and execution.

You will balance hands-on coding in C++ and Python with organizational leadership, shaping roadmaps, mentoring engineers, and delivering scalable ML pipelines in a fast-paced environment.