Staff ML Engineer - 3D Perception & Multimodal Models

Neura Market; Mountain View (CA); Hybrid; USD 251,000 - 310,000; Full time

Job summary

Waymo is seeking a research scientist to advance multimodal LLMs and world models for 3D perception using camera, LiDAR and radar data. This hybrid role reports to a Principal Research Scientist and collaborates with engineering and research teams across Waymo.

The role emphasizes scalable training, distributed inference, and efficient model distillation, with hands-on work on Transformer architectures and cutting-edge ML techniques.

Qualifications

• PhD or Masters in Computer Science, ML, Robotics or related field with 4+ years RL/Foundation Models.
• Experience implementing scalable distributed training flows (Data parallel, FSDP, sharding).
• Proficiency in JAX, Flax, and optionally TensorFlow/PyTorch.
• Willingness to work with globally distributed inference infrastructure.
• Hands-on experience optimizing training and inference of Transformer architectures.

Responsibilities

• Conduct experimentation to train and deploy state-of-the-art Multimodal LLMs and World models for 3D Perception.
• Partner with engineering and research teams to deploy models and workflows for continuous training.
• Apply quantization, pruning, knowledge distillation, and efficient attention mechanisms.
• Develop scalable data pipelines for Training & Eval from multiple sources.
• Design evaluation frameworks for perception models.
• Build infrastructure for large-scale model distillation and bulk-inference pipelines.
• Experiment with model partitioning and sharding to improve scalability.
• Build and maintain tools for performance analysis, profiling and debugging ML models.

Skills

Transformer optimization

World models

Quantization

Pruning

Knowledge distillation

Inference infrastructure

Sharding

Distributed training

Education

PhD in Computer Science, Machine Learning or Robotics

Masters in Computer Science, Machine Learning or Robotics

Tools

JAX

Flax

TensorFlow/PyTorch

Data parallel

FSDP

XManager

Job description

Waymo is seeking a research scientist to advance multimodal LLMs and world models for 3D perception using camera, LiDAR and radar data. This hybrid role reports to a Principal Research Scientist and collaborates with engineering and research teams across Waymo.

The role emphasizes scalable training, distributed inference, and efficient model distillation, with hands-on work on Transformer architectures and cutting-edge ML techniques.