Engineering Manager, ML Performance

Google Inc.; Sunnyvale (CA); On-site; USD 207,000 - 301,000; Full time

Benefits offered by this job

Health, dental, vision, life, and disability insurance

401(k) retirement benefits with company match

Paid Time Off: 20 days vacation per year

Sick Time: 40 hours/year + discretionary days

Job summary

Google Inc. is seeking an experienced professional for a software development role in Sunnyvale, CA, focused on machine learning design and technical leadership. Responsibilities include leading benchmarks in ML training, optimizing TPU performance, and driving innovations. A Bachelor’s degree and extensive experience in software development and ML are essential.

The position offers a competitive compensation of $207,000 - $301,000 plus bonuses and equity. Join Google to innovate in the ML domain.

Qualifications

• 8 years of experience in software development.
• 5 years of experience in ML design and optimization.
• Experience in a technical and people management role.

Responsibilities

• Lead a team focused on ML training and serving benchmarks.
• Achieve performance for customer launches and benchmark submissions.
• Co-design TPU-friendly models for enhanced performance.

Skills

Software Development

Machine Learning Design

Technical Leadership

Performance Analysis

Education

Bachelor’s degree or equivalent experience

Tools

CUDA

JAX

OpenXLA

Job description

Benefits

• Health, dental, vision, life, disability insurance
• Retirement Benefits: 401(k) with company match
• Paid Time Off: 20 days of vacation per year, accruing at a rate of 6.15 hours per pay period for the first five years of employment
• Sick Time: 40 hours/year (increased to 69 hours/year for Seattle) including 5 discretionary sick days per instance
• Maternity Leave (Short-Term Disability + Baby Bonding): 28-30 weeks
• Baby Bonding Leave: 18 weeks
• Holidays: 13 paid days per year

Note: By applying to this position you will have an opportunity to share your preferred working location from the following: Sunnyvale, CA, USA; Kirkland, WA, USA.

Requirements

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 3 years of experience in a technical leadership role.
• 2 years of experience in a people management or team leadership role.
• Experience with ML performance analysis, benchmarking, and computer architecture.

Preferred Qualifications

• Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
• 3 years of experience working in a complex, matrixed organization involving cross‑functional or cross‑business projects.
• Experience in ML accelerators (GPUs, TPUs) and low‑level kernel programming/tuning using tools like CUDA, Triton, or Pallas.
• Experience with compiler optimization (MLIR, OpenXLA) and integrating frameworks/serving libraries (PyTorch, JAX, vLLM) to maximize hardware efficiency.
• Ability to adapt ML models to specific hardware strengths and use performance benchmarking to guide both optimization and future hardware design.

Responsibilities

• Lead a team of software engineers focused on identifying and maintaining ML training and serving benchmarks that are representative to Google production and the broader ML industry.
• Achieve performance for customer launches, and for engaged benchmark submissions (ML Commons, InferenceX, etc.).
• Use benchmarks to identify performance opportunities and drive both near‑term state‑of‑the‑art kernel development and out‑of‑the‑box performance (compiler/runtime optimizations, agentic tooling, auto‑sharding) directly and in collaboration with partner teams.
• Participate in algorithmic innovations exploiting new TPU hardware features and model‑preserving optimizations (speculative decoding, sparsity, quantization, LoRA, etc.).
• Co‑design models that are TPU‑friendly to showcase model quality and performance advancement for open‑source models typically designed on GPUs.

Compensation

US: $207,000 - $301,000 (USD) + 20% bonus target + equity + benefits.

Equal Opportunity Employer

Google is proud to be an equal opportunity and affirmative action employer. We are committed to building a workforce that is representative of the users we serve, creating a culture of belonging, and providing an equal employment opportunity regardless of race, creed, color, religion, gender, sexual orientation, gender identity or expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition (including breastfeeding), expecting or parents‑to‑be, criminal histories consistent with legal requirements, or any other basis protected by law. See also Google’s EEO Policy, Know your rights: workplace discrimination is illegal, Belonging at Google, and How we hire.

Google is a global company and, in order to facilitate efficient collaboration and communication globally, English proficiency is a requirement for all roles unless stated otherwise in the job posting.