Senior ML Engineer: TPU Training & Inference Optimizer

Google LLC; Singapore; On-site; SGD 150,000 - 240,000; Full time

Job summary

Google LLC in Singapore seeks a Senior Software Engineer specializing in Machine Learning and TPU workload optimization. This role focuses on onboarding workloads to production, optimizing training and inference, and collaborating with ML research and performance teams.

A strong background in ML systems and large-scale deployments is essential. The successful candidate will contribute to cutting-edge AI infrastructure, enabling efficient model training and inference on Google Cloud hardware,

Qualifications

• Bachelor's degree or equivalent practical experience required.
• 5+ years software development in one or more languages.
• 2+ years leading ML design and optimizing ML infrastructure.
• 1 year of experience with training/inference techniques (Megatron-LM, DeepSpeed, TensorRT-LLM, etc.).

Responsibilities

• Act as ML software engineer leading onboarding for stable stack and optimization for training and serving on TPU NPIs.
• Architect training/inference onboarding and optimization frameworks for customer needs.
• Collaborate with ML research and optimization teams.
• Ship stable stack to customers for training/inference optimization on TPU.
• Prepare and optimize very large reference models for SOTA single- and multi-host inference at scale.

Skills

Software development

ML design leadership

ML training/inference

Education

Bachelor's degree or equivalent practical experience

Tools

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

Job description

Google LLC in Singapore seeks a Senior Software Engineer specializing in Machine Learning and TPU workload optimization. This role focuses on onboarding workloads to production, optimizing training and inference, and collaborating with ML research and performance teams.

A strong background in ML systems and large-scale deployments is essential. The successful candidate will contribute to cutting-edge AI infrastructure, enabling efficient model training and inference on Google Cloud hardware,