Staff ML Engineer: TPU Training & Inference Optimizer

Google LLC; Singapore; On-site; SGD 180,000 - 320,000; Full time

Job summary

Google LLC in Singapore is seeking a Staff Software Engineer to advance ML training and TPU workload optimization across production training and inference pipelines. You will lead onboarding of customer workloads to scalable production environments on TPU and GPU hardware.

The role emphasizes collaboration with ML research and performance teams to build robust tooling and achieve high-throughput, low-latency ML services at scale.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of software development experience.
• 4 years leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 2 years of experience with state of the art training (e.g. Megatron-LM, DeepSpeed) and inference techniques (e.g. TensorRT-LLM, vLLM, SGLang).

Responsibilities

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs.
• Set the technical direction and architect the training/inference onboarding and optimization frameworks for our customer needs. Work closely with customers to onboard their workloads to production and optimization.
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating state of the art single-host and multi-host inference solutions, especially at large scale.

Skills

Software development

ML design leadership

ML infrastructure optimization

Model deployment

Model evaluation

Data processing

Debugging

Fine tuning

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

TPUs

GPUs

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in Engineering, Computer Science, or related field

Tools

TPUs

GPUs

HPC

Job description

Google LLC in Singapore is seeking a Staff Software Engineer to advance ML training and TPU workload optimization across production training and inference pipelines. You will lead onboarding of customer workloads to scalable production environments on TPU and GPU hardware.

The role emphasizes collaboration with ML research and performance teams to build robust tooling and achieve high-throughput, low-latency ML services at scale.