Staff ML Engineer - TPU Training & Inference Optimizer

Google; Singapore; On-site; SGD 180,000 - 300,000; Full time

Job summary

Google is hiring software engineers to advance AI training and inference workloads on Cloud GPUs and TPUs. You will lead onboarding, optimize training and serving stacks, and collaborate with ML research teams to push the boundaries of large-scale models.

The role emphasizes architecture, collaboration with customers, and delivering production-grade, scalable ML solutions for Google Cloud customers and Googlers worldwide.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 4 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 2 years of experience with state of the art training (e.g. Megatron-LM, DeepSpeed) and inference techniques (e.g. TensorRT-LLM, vLLM, SGLang).

Responsibilities

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs.
• Set the technical direction and architect the training/inference onboarding and optimization frameworks for our customer needs. Work closely with customers to onboard their workloads to production and optimization.
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating state of the art single‑host and multi‑host inference solutions, especially at large scale.

Skills

Software development

ML design

ML optimization

Model training

Model inference

Education

Bachelor's degree or equivalent

Tools

Megatron-LM

DeepSpeed

TensorRT

vLLM

SGLang

Job description

Google is hiring software engineers to advance AI training and inference workloads on Cloud GPUs and TPUs. You will lead onboarding, optimize training and serving stacks, and collaborate with ML research teams to push the boundaries of large-scale models.

The role emphasizes architecture, collaboration with customers, and delivering production-grade, scalable ML solutions for Google Cloud customers and Googlers worldwide.