TPU ML Engineer: Optimize Large-Scale Training

Visa Hunt; Singapore; On-site; SGD 120,000 - 170,000; Full time

Job summary

Google's AI and Infrastructure team seeks a software engineer to advance the ML stack on TPU NPIs, shaping training and serving solutions at scale and reliability for global services.

You will contribute across firmware, kernel, and server integration, leading design reviews and driving components from concept to launch in a fast-paced, collaborative environment. Prior hands-on ML deployment and optimization experience is highly valued.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 2 years of experience with software development in one or more programming languages, or 1 year of experience with an advanced degree.
• 1 year of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 1 year of experience with training (e.g., Megatron-LM, DeepSpeed) and inference techniques (e.g., TensorRT-LLM, vLLM, SGLang).

Responsibilities

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs.
• Possess extensive technical knowledge of firmware design, kernel development, and server system integration, with the ability to influence the ecosystem.
• Triage product or system issues and debug/track/resolve by analyzing the sources of issues and the impact on hardware, network, or service operations and quality.
• Participate in, or lead design reviews with peers and stakeholders to decide amongst available technologies.
• Lead the server software design, development, and implementation on a scalable and high-quality Cloud Server Program, overseeing the project from conception to launch.

Skills

Programming languages

ML design

Education

Bachelor's degree or equivalent

Tools

Megatron-LM

DeepSpeed

TensorRT-LLM

vLLM

SGLang

Job description

Google's AI and Infrastructure team seeks a software engineer to advance the ML stack on TPU NPIs, shaping training and serving solutions at scale and reliability for global services.

You will contribute across firmware, kernel, and server integration, leading design reviews and driving components from concept to launch in a fast-paced, collaborative environment. Prior hands-on ML deployment and optimization experience is highly valued.