Staff ML Engineer: Multimodal Perception & 3D World Models

Waymo; Mountain View (CA); Hybrid; USD 251,000 - 310,000; Full time

Job summary

Waymo seeks a research scientist to advance multimodal large language models and world models for 3D perception across camera, LiDAR, and radar data. You will collaborate with engineering and research teams to deploy new models, design scalable data pipelines, and build evaluation frameworks for perception models.

A PhD or Masters with 4+ years in RL or foundation models is required. You will work in a hybrid environment reporting to a Principal Research Scientist, with responsibilities spanning

Qualifications

• PhD or Masters in Computer Science, ML, Robotics, or similar field with 4+ years RL/Foundation Models experience.
• Experience building scalable, distributed training flows (Data parallel, FSDP, sharding).
• Proficiency with JAX/Flax and PyTorch or TensorFlow.
• Comfort with globally distributed inference infrastructure.
• Hands-on experience optimizing Transformer training and inference.

Responsibilities

• Conduct experiments to train and deploy state-of-the-art Multimodal LLMs and World models for 3D perception.
• Collaborate with engineering and research teams to deploy models and streamline training on new data.
• Apply quantization, pruning, knowledge distillation, and efficient attention techniques.
• Develop scalable data pipelines for training and evaluation from multiple sources.
• Design evaluation frameworks for perception models.
• Build infrastructure for large-scale model distillation and bulk inference.
• Experiment with model partitioning and sharding to improve scalability.
• Create and maintain tools for performance analysis, profiling, and debugging.

Skills

Distributed training

Data parallel

Transformer optimization

Multi-Modal learning

Reinforcement Learning

Foundation Models

Generative models

Diffusion models

Autoregressive models

JAX

Flax

TensorFlow

PyTorch

Gemax

XManager

Education

PhD or Masters in CS/ML/Robotics

Tools

JAX

Flax

TensorFlow

PyTorch

Gemax

XManager

Job description

Waymo seeks a research scientist to advance multimodal large language models and world models for 3D perception across camera, LiDAR, and radar data. You will collaborate with engineering and research teams to deploy new models, design scalable data pipelines, and build evaluation frameworks for perception models.

A PhD or Masters with 4+ years in RL or foundation models is required. You will work in a hybrid environment reporting to a Principal Research Scientist, with responsibilities spanning