Staff Machine Learning Engineer (Generative Model Performance & Efficiency)

Waymo LLC; New York (NY); Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Vision insurance

401(k)

Equity opportunities

Hybrid work model

Remote work opportunities

Paid time off

Job summary

Waymo LLC's Simulator Team in New York seeks a senior ML engineer to optimize large-scale generative and diffusion models for driver simulation. You will analyze architectures, reduce training and inference time, and implement advanced quantization and pruning techniques across TPU/GPU targets.

You will collaborate with ML researchers and engineers to push 3D world modeling, maintain profiling tools, and improve serving latency in real-time simulation workloads.

Qualifications

• 5+ years of experience with deep learning architectures (Transformers, Diffusion Models, MoEs) and optimization techniques.
• Strong Python and C++ programming with software engineering best practices.
• Hands-on experience with quantization, pruning, distillation, and other model compression methods.
• Familiarity with ML compilers like XLA.
• Experience profiling ML workloads with XProf, Perfetto, Nsight; familiarity with TPUs/GPUs.

Responsibilities

• Analyze model architectures and identify bottlenecks in training and inference performance.
• Apply and develop quantization, pruning, knowledge distillation, and efficient attention mechanisms.
• Optimize model code for TPUs and GPUs, leveraging XLA and low-level libraries.
• Experiment with partitioning and sharding strategies to improve scalability and efficiency.
• Design and implement low-latency, high-throughput serving for generative models and ML workloads.

Skills

Python

C++

ML frameworks (TensorFlow/PyTorch)

JAX/Flax

Profiling tools

XLA

TPU/GPU optimization

Software engineering practices

Education

MS or PhD in Computer Science / ML / Robotics

Tools

JAX

Flax

TensorFlow

PyTorch

XLA

NVIDIA Nsight

Perfetto

XProf

XManager

Gemax

Job description

• The Simulator Team at Waymo builds state-of-the-art simulations of realistic environments for testing, training, and validation of the Waymo Driver. Our team is a diverse, and collaborative group of machine learning (ML) engineers, software engineers, and ML research engineers
• We develop industry-leading simulation solutions using advanced generative and reconstructive ML algorithms, to model the real world, encompassing realistic agents, roads, traffic systems, weather, and the full sensor suite (Camera, Lidar, Radar)
• To accelerate the fidelity, scalability, controllability, and richness of our simulations, we are pushing the frontiers of 3D world modeling. We leverage state-of-the-art ML technologies trained on large-scale datasets to create dynamic and semantically rich virtual worlds, directly impacting the development and validation of the Waymo Driver
• In this role, you will report to a Senior Staff Engineering Manager
• Analyze model architectures and identify bottlenecks in training and inference performance (e.g., memory bandwidth, compute, communication)
• Apply and develop techniques such as quantization (e.g., FP8, INT4), pruning, knowledge distillation, and efficient attention mechanisms
• Optimize model code for specific hardware accelerators (TPUs, GPUs), leveraging compiler features and low-level libraries (e.g., XLA)
• Experiment with different model partitioning and sharding strategies (e.g., data, tensor, pipeline parallelism, expert parallelism) to improve scalability and efficiency
• Design and implement low-latency, high-throughput serving solutions for generative models and optimize training pipelines to reduce training time
• Build and maintain tools for performance analysis, profiling (e.g., xprof), and debugging of ML models

Benefits

• Medical, dental, and vision insurance for employees and dependents
• Employee assistance programs focused on mental health
• Personalized workplace adjustments for diverse needs and abilities, including physical, mental, and neurodivergent considerations
• Access to mental health apps
• Onsite wellness centers
• Support programs including menopause benefit
• Medical advocacy program for transgender employees
• Second medical opinion for you and your loved ones
• Counseling services
• Competitive compensation
• Regular bonus and equity performance grant opportunities
• Generous 401(k) and regional retirement plans
• 1-on-1 financial coaching
• Annual cross-company compensation review and pay equity analysis
• Fertility and growing family assistance
• Parental leave and baby bonding leave
• Survivor income benefit
• Backup childcare
• Elder care and support
• Caregiver leave
• Paid time off, including vacation, bereavement, sick leave, parental leave, disability, and holidays
• Jury duty leave
• Military leave
• Hybrid work model with remote work opportunities also available
• Educational reimbursement
• Peer learning and coaching platform
• Donation matching programs
• Volunteer hours
• Employee resource groups
• Internal community groups and local culture clubs
• Inspiring spaces to work, recharge, and collaborate with fellow Waymonauts
• On-site meals and snacks
• Fitness centers, massage programs, and ergonomic support
• On-demand fitness, wellbeing, and cooking classes
• Commuter benefits

Strong programming skills in Python and potentially C++, with experience in software development best practicesProficiency in JAX, Flax, and potentially TensorFlow/PyTorch5+ years of experience with deep learning architectures (especially Transformers, Diffusion Models, MoEs), algorithms, and optimization techniquesMS or PhD in Computer Science, Machine Learning, Robotics, or a related fieldHands-on experience with quantization, pruning, distillation, and other model compression methodsExpertise in using profiling tools (e.g., XProf, Perfetto, NVIDIA Nsight) to diagnose performance issues in ML workloadsFamiliarity with ML compilers like XLA and an understanding of how they translate high-level code to efficient hardware instructionsKnowledge of TPU and GPU architectures and how to optimize code for themExperience contributing to frameworks and libraries that improve training speed and scalability (e.g., JAX, Gemax, XManager)Understanding of concepts related to training and serving models across multiple devices and machines