Staff ML Systems Engineer - GPU & Performance

Google; Sunnyvale (CA); On-site; USD 207,000 - 300,000; Full time

Benefits offered by this job

Equity

Performance bonus

Job summary

Google is seeking a senior software engineer for the Core ML GPU Performance team in Sunnyvale to optimize and model GPU systems for large-scale ML workloads. You will help drive performance improvements across production and research pipelines, collaborating with cross-functional teams to scale LLMs on accelerator hardware.

You will work on architecture-level optimizations, benchmarking, and tooling to improve efficiency and throughput, shaping the future of ML infrastructure at Google.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• Experience with modern GPU architectures, memory hierarchies, and performance bottlenecks.
• Experience with low-level GPU programming (CUDA, Triton, CUTLASS, etc.) and performance engineering techniques.
• Experience with modern LLMs and their deployment on AI accelerators.

Responsibilities

• Identify and optimize LLM training and serving benchmarks; drive XLA:GPU/Triton performance and guide XLA releases.
• Partner with product teams to onboard, optimize, and scale LLMs on GPU hardware.
• Conduct architecture-level simulations, performance benchmarking, and roofline analyses to guide system designs.
• Analyze fleet-wide performance and efficiency metrics to identify bottlenecks and engineer scalable optimizations.
• Research and implement model/data efficiency techniques and profiling mechanisms to improve workload performance.

Skills

Software development

ML design & infrastructure

GPU architectures

Low-level GPU programming

LLM deployment

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD (preferred)

Tools

OpenXLA

Job description

Google is seeking a senior software engineer for the Core ML GPU Performance team in Sunnyvale to optimize and model GPU systems for large-scale ML workloads. You will help drive performance improvements across production and research pipelines, collaborating with cross-functional teams to scale LLMs on accelerator hardware.

You will work on architecture-level optimizations, benchmarking, and tooling to improve efficiency and throughput, shaping the future of ML infrastructure at Google.