Senior ML Engineer - TPU Training & Inference Optimizer

Google India; Singapore; On-site; SGD 180,000 - 340,000; Full time

Job summary

Google is hiring ML software engineers to lead onboarding and optimization for training and serving workloads on latest TPU NPIs. You will collaborate with ML research and performance teams to craft scalable inference solutions and deploy reference models at scale.

The role focuses on building stable ML infrastructure, enabling customers to run efficient training and inference, and optimizing frameworks for latency and throughput on cloud GPUs/TPUs.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 2 years of experience leading ML design and optimizing ML infrastructure.
• 1 year of experience with training and inference techniques.

Responsibilities

• Act as ML software engineer leading onboarding for stable stack and optimization for training and serving solutions on TPU NPIs.
• Work on the technical details and architect the training/inference onboarding and optimization frameworks for customer needs.
• Collaborate with ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models for SOTA single-host and multi-host inference at large scale.

Skills

Software development

ML design

Training & inference techniques

Education

Bachelor’s degree or equivalent practical experience

Tools

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

Job description

Google is hiring ML software engineers to lead onboarding and optimization for training and serving workloads on latest TPU NPIs. You will collaborate with ML research and performance teams to craft scalable inference solutions and deploy reference models at scale.

The role focuses on building stable ML infrastructure, enabling customers to run efficient training and inference, and optimizing frameworks for latency and throughput on cloud GPUs/TPUs.