Staff ML Engineer: TPU Training & Inference Optimizer

GOOGLE ASIA PACIFIC PTE. LTD.; Singapore; On-site; SGD 180,000 - 280,000; Full time

Job summary

Google Asia Pacific Pte. Ltd. in Singapore is seeking a seasoned ML software engineer to lead onboarding for stable training and inference pipelines on TPUs and GPUs at scale. You will collaborate with ML research and performance teams to design end-to-end solutions and onboard workloads.

The role requires 8 years in software development, 4 years leading ML design, and experience with Megatron-LM, DeepSpeed, and TensorRT-based frameworks, in a fast-paced production environment.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 8 years of experience in software development.
• 4 years of experience leading ML design and optimizing ML infrastructure.
• 2 years of experience with state of the art training and inference techniques.

Responsibilities

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs.
• Set the technical direction and architect the training/inference onboarding and optimization frameworks for our customer needs.
• Work closely with ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating state of the art single-host and multi-host inference solutions.

Skills

Software development experience

ML design leadership

ML infrastructure optimization

Onboarding ML workloads

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in Engineering or CS

Tools

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

TPUs

GPUs

Job description

Google Asia Pacific Pte. Ltd. in Singapore is seeking a seasoned ML software engineer to lead onboarding for stable training and inference pipelines on TPUs and GPUs at scale. You will collaborate with ML research and performance teams to design end-to-end solutions and onboard workloads.

The role requires 8 years in software development, 4 years leading ML design, and experience with Megatron-LM, DeepSpeed, and TensorRT-based frameworks, in a fast-paced production environment.