Staff ML Engineer, Perception Research

Socket.dev; Mountain View (CA); Hybrid; USD 251,000 - 310,000; Full time

Benefits offered by this job

Health, dental, vision

401(k) with company match

Paid time off

Sick time

Maternity leave

Job summary

Waymo is seeking a PhD/MS holder to advance multimodal learning and large-scale model deployment. The role focuses on training state-of-the-art models for 3D perception using camera, LiDAR, and radar data.

Collaboration with cross-functional teams will drive production-grade workflows for continuous training and evaluation. The candidate will optimize training and inference of transformer architectures, explore scaling strategies, and help build robust tools for performance analysis and

Qualifications

• PhD or Masters in Computer Science, ML, Robotics or related field.
• 4+ years of industry or post-doc research in Reinforcement Learning or Foundation Models.
• Experience building scalable model training pipelines.

Responsibilities

• Conduct experiments to train and deploy state-of-the-art Multimodal LLMs and World models for 3D Perception.
• Collaborate with engineering and research teams to deploy models and create workflows for continuous training.
• Develop and optimize techniques like quantization, pruning, distillation, and efficient attention.

Skills

PhD or Masters

4+ years RL/FM research

JAX/Flax

TensorFlow/PyTorch

Distributed training

Education

PhD or MSc in CS/ML/Robotics

Tools

JAX/Flax

TensorFlow

PyTorch

Job description

Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The World's Most Experienced Driver™—to improve access to mobility while saving thousands of lives now lost to traffic crashes. The Waymo Driver powers Waymo’s fully autonomous ride-hail service and can also be applied to a range of vehicle platforms and product use cases. The Waymo Driver has provided over ten million rider-only trips, enabled by its experience autonomously driving over 100 million miles on public roads and tens of billions in simulation across 15+ U.S. states.

Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The World's Most Experienced Driver™—to improve access to mobility while saving thousands of lives now lost to traffic crashes. The Waymo Driver powers Waymo’s fully autonomous ride-hail service and can also be applied to a range of vehicle platforms and product use cases. The Waymo Driver has provided over ten million rider-only trips, enabled by its experience autonomously driving over 100 million miles on public roads and tens of billions in simulation across 15+ U.S. states.

This role follows a hybrid work schedule and reports to a Principal Research Scientist.

You will

• Conduct comprehensive experimentation to train and deploy state-of-the-art Multimodal LLMs and World models to perform 3D Perception using sensor information from Camera, LiDAR and Radar..
• Partner effectively with engineering and research teams across Waymo to deploy new models, and implement efficient workflows for model development and continuous training on new front-filled data.
• Apply and develop techniques such as quantization, pruning, knowledge distillation, and efficient attention mechanisms.
• Develop and maintain scalable data pipelines for Training & Eval to process data from multiple sources.
• Design and implement evaluation frameworks for perception models.
• Develop infrastructure for large-scale model distillation and bulk-inference pipelines for teacher models.
• Experiment with different model partitioning and sharding strategies to improve scalability and efficiency.
• Build and maintain tools for performance analysis, profiling (e.g., xprof), and debugging of ML models.

You have

• PhD or Masters in Computer Science, Machine Learning, Robotics, or a similar technical field, with 4+ years of industry or post-doc research experience in Reinforcement Learning or Foundation Models.
• Proficiency in implementing model training flows in a scalable, distributed and performant manner such as Data parallel, FSDP and other sharding approaches.
• Proficiency in JAX, Flax, and potentially TensorFlow/PyTorch.
• A willingness to work with complexity of globally distributed inference infrastructure.
• Hands on experience with optimizing the training and inference of Transformer architectures

We prefer

• PhD in Computer Science, Machine Learning, or Robotics, with a research focus on Reinforcement Learning, Foundation Models, or Multi-Modal learning.
• Substantial involvement in and contributions to high impact industry AI projects.
• Experience in generative models for domains such as world models, images, videos, 3D, using techniques such as diffusion or autoregressive models.
• Experience contributing to frameworks and libraries that improve training speed and scalability (e.g., JAX, Gemax, XManager)

Disclosure for WA Based & Remote Roles

In accordance with Washington state law, we are highlighting our comprehensive benefits package, which is available to all eligible US based employees. Benefits for this role include:

• Health, dental, vision, life, disability insurance
• Retirement Benefits: 401(k) with company match
• Paid Time Off: 20 days of vacation per year, accruing at a rate of 6.15 hours per pay period for the first five years of employment
• Sick Time: 40 hours/year (statutory, where applicable); 5 days/event (discretionary)
• Maternity Leave (Short-Term Disability + Baby Bonding): 28-30 weeks
• Baby Bonding Leave: 18 weeks
• Holidays: 13 paid days per year

The expected base salary range for this full-time position across US locations is listed below. Actual starting pay will be based on job-related factors, including exact work location, experience, relevant training and education, and skill level.

Salary Range

$251,000—$310,000 USD

Waymo employees are also eligible to participate in Waymo’s discretionary annual bonus program, equity incentive plan, and generous Company benefits program, subject to eligibility requirements.