Staff Research Scientist, Multimodal Perception WorldModels

Neura Market; Mountain View (CA); Hybrid; USD 251,000 - 310,000; Full time

Benefits offered by this job

Health insurance

401(k) with company match

Paid time off - 20 days/year

Sick time - 40 hours/year

Maternity leave (Short-Term Disability

Baby bonding leave

13 paid holidays per year

Job summary

Waymo, an autonomous driving technology company, seeks researchers to advance multimodal LLMs for 3D perception using camera, LiDAR, and radar. The role emphasizes scalable training and evaluation, collaboration across teams, and dissemination of findings to the broader community.

Ideal candidates hold a PhD or Master’s in a technical field with 4+ years in reinforcement learning or foundation models, and a track record of publications or open-source contributions.

Qualifications

• PhD or Masters in Computer Science, ML, Robotics, with 4+ years of RL or foundation models research.
• Demonstrated high-impact publications or significant open-source contributions.
• Experience implementing scalable distributed model training and inference infrastructure.

Responsibilities

• Research & develop state-of-the-art multimodal LLMs for 3D perception using camera, LiDAR and radar data.
• Integrate research from the broader AI community into Waymo’s encoders and sensor understanding models.
• Collaborate with engineering and research teams to share recipes and best practices.
• Develop scalable data pipelines for training and evaluation across multiple sources.
• Design and implement evaluation frameworks for perception models.
• Study model behaviors, scaling efficacy, and design ablations.
• Model perception solutions to understand sensor information from autonomous vehicles.
• Communicate findings via technical reports and publications.

Skills

Reinforcement Learning

Foundation Models

Distributed Training

Publications

Education

PhD or Masters in CS/ML/Robotics

Tools

Data parallel

FSDP

Model sharding

Job description

Waymo, an autonomous driving technology company, seeks researchers to advance multimodal LLMs for 3D perception using camera, LiDAR, and radar. The role emphasizes scalable training and evaluation, collaboration across teams, and dissemination of findings to the broader community.

Ideal candidates hold a PhD or Master’s in a technical field with 4+ years in reinforcement learning or foundation models, and a track record of publications or open-source contributions.