Senior ML Engineer — TPU Training & Inference Optimization

Google LLC; Singapore; On-site; SGD 180,000 - 240,000; Full time

Job summary

Google LLC is seeking a Senior Software Engineer specializing in Machine Learning and TPU workload optimization in Singapore. The role focuses on onboarding customers to stable training and inference stacks on the latest TPU technologies, and optimizing performance across large-scale models.

The candidate will collaborate with ML research and optimization teams to build scalable infrastructure, enabling efficient deployment and inference on cloud GPUs/TPUs while maintaining high reliability for

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 2 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 1 year of experience with training (e.g., Megatron-LM, DeepSpeed) and inference techniques (e.g., TensorRT-LLM, vLLM, SGLang).

Responsibilities

• Act as ML software engineer leading onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs.
• Work on the technical details and architect the training/inference onboarding and optimization frameworks for our customer needs.
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating SOTA single-host and multi-host inference solutions, especially at large scale.

Skills

Software development

ML design leadership

ML infrastructure optimization

Training & inference techniques

Education

Bachelor’s degree or equivalent practical experience

Master’s degree or PhD in Computer Science or related field

Tools

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

Job description

Google LLC is seeking a Senior Software Engineer specializing in Machine Learning and TPU workload optimization in Singapore. The role focuses on onboarding customers to stable training and inference stacks on the latest TPU technologies, and optimizing performance across large-scale models.

The candidate will collaborate with ML research and optimization teams to build scalable infrastructure, enabling efficient deployment and inference on cloud GPUs/TPUs while maintaining high reliability for