Staff ML Engineer: TPU Training & Inference Optimization

Google; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

Google Singapore is seeking a Staff Software Engineer in Machine Learning for TPU workload optimization to advance training and inference at scale.

You will own onboarding for the stable stack, set technical direction, collaborate with ML research and performance teams, and ship optimized solutions on TPU NPIs for production workloads.

The role requires extensive ML experience and a track record of delivering high-performance systems; interviews are typically in-person in Singapore.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 8 years of experience in software development.
• 4 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 2 years of experience with state of the art training (e.g. Megatron-LM, DeepSpeed) and inference techniques (e.g. TensorRT-LLM, vLLM, SGLang).

Responsibilities

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs.
• Set the technical direction and architect the training/inference onboarding and optimization frameworks for our customer needs.
• Work closely with customers to onboard their workloads to production and optimization.
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating state of the art single-host and multi-host inference solutions, especially at large scale.

Skills

Software development

ML design

ML optimization

Model deployment

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

Education

Bachelor's degree or equivalent practical experience

Tools

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

Job description

Google Singapore is seeking a Staff Software Engineer in Machine Learning for TPU workload optimization to advance training and inference at scale.

You will own onboarding for the stable stack, set technical direction, collaborate with ML research and performance teams, and ship optimized solutions on TPU NPIs for production workloads.

The role requires extensive ML experience and a track record of delivering high-performance systems; interviews are typically in-person in Singapore.