Principal ML Performance GPU Engineer

Google LLC; Sunnyvale (CA); On-site; USD 262,000 - 364,000; Full time

Job summary

Google LLC Sunnyvale, CA, seeks a Senior Staff Software Engineer in ML Performance for GPUs to lead end-to-end ML optimization and drive Gemini model performance on accelerator hardware. You will work across teams to push peak efficiency and scale across large deployments.

You will guide headroom analysis, develop custom kernels, and advance XLA GPU releases while mentoring teams through complex projects in a fast-paced environment.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 7 years of experience leading technical project strategy, ML design and optimizing end-to-end Machine Learning (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• Experience with modern GPU architectures, memory hierarchies, and performance bottlenecks.
• Experience with low-level GPU programming (CUDA, Triton, CUTLASS, etc.) and performance engineering techniques.
• Experience with modern LLMs and their deployment on AI accelerators.

Responsibilities

• Identify and maintain LLM training and serving benchmarks, use these to identify performance opportunities, drive XLA:GPU/Triton performance toward state-of-the-art, and guide XLA releases.
• Collaborate with DeepMind to solve their ML model performance challenges, such as onboarding new LLM models and products on GPU hardware and enabling LLMs to train and serve efficiently on a very large scale.
• Run architecture level simulations on GPU designs, perform roofline analysis to guide internal teams, and explore novel model/data efficiency techniques.
• Analyze performance and efficiency metrics to identify bottlenecks, design, and implement solutions at Google fleet-wide scale.
• Collaborate with tooling and fleet metrics sub-teams to build tools that track performance and efficiency and extract metrics from Google’s running workloads.

Skills

ML design

Technical leadership

Performance optimization

End-to-end ML deployment

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in Engineering, CS, or related field

Tools

CUDA

Triton

CUTLASS

Job description

Google LLC Sunnyvale, CA, seeks a Senior Staff Software Engineer in ML Performance for GPUs to lead end-to-end ML optimization and drive Gemini model performance on accelerator hardware. You will work across teams to push peak efficiency and scale across large deployments.

You will guide headroom analysis, develop custom kernels, and advance XLA GPU releases while mentoring teams through complex projects in a fast-paced environment.