Senior ML Engineer - TPU Training & Inference Optimization

GOOGLE ASIA PACIFIC PTE. LTD.; Singapore; On-site; SGD 150,000 - 210,000; Full time

Job summary

GOOGLE ASIA PACIFIC PTE. LTD. is seeking an ML software engineer to lead onboarding for a stable stack, optimize training and serving solutions on the latest TPU NPIs, and work closely with customers to onboard workloads to production and optimization.

You will collaborate with ML research, ML performance, model optimization tooling, and other teams to ship a stable stack for training/inference optimization, including large reference models and scalable inference at scale.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 2 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 1 year of experience with training (e.g., Megatron-LM, DeepSpeed) and inference techniques (e.g., TensorRT-LLM, vLLM, SGLang).

Responsibilities

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs. Work closely with customers to onboard their workloads to production and optimization.
• Work on the technical details and architect the training/inference onboarding and optimization frameworks for our customer needs.
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating SOTA single-host and multi-host inference solutions, especially at large scale.

Skills

ML software development

Leadership

Training/inference optimization

Model deployment

Education

Bachelor's degree

Master's/PhD

Tools

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

Job description

GOOGLE ASIA PACIFIC PTE. LTD. is seeking an ML software engineer to lead onboarding for a stable stack, optimize training and serving solutions on the latest TPU NPIs, and work closely with customers to onboard workloads to production and optimization.

You will collaborate with ML research, ML performance, model optimization tooling, and other teams to ship a stable stack for training/inference optimization, including large reference models and scalable inference at scale.