Senior Multimodal ML Engineer - Vision & Foundation Models

Waymo; California (MO); Hybrid; USD 213,000 - 263,000; Full time

Job summary

Waymo is seeking a Senior ML Engineer in the Semantics org to develop and deploy state-of-the-art computer vision and multimodal models. You will build an AI agent framework that integrates large foundation models with perception outputs and knowledge bases, and apply RL and fine-tuning to improve captioning and reasoning.

You will collaborate with ML Infra, Perception, Behavior, and AI Foundation teams, leading complex projects in a fast-paced R&D environment, and own the full lifecycle from

Qualifications

• Master’s degree in Computer Science or a related technical field.
• 4+ years of hands-on experience training and shipping deep learning models for computer vision.
• 1+ years of experience with LLMs or VLMs, including fine-tuning or prompting.
• Strong software engineering fundamentals and scalable systems design.
• Experience building large-scale ML data pipelines.

Responsibilities

• Develop and train state-of-the-art CV/multimodal models.
• Design scalable AI agent framework integrating foundation models.
• Apply fine-tuning and RL to improve captioning and reasoning.
• Prototype prompting strategies for Vision-Language Models.
• Collaborate with ML Infra, Perception, Behavior, and AI Foundation teams.
• Own full system lifecycle from development to production deployment.

Skills

Python

PyTorch

JAX

TensorFlow

Computer Vision

LLMs

VLMs

RAG

System design

Education

Master’s degree in Computer Science or related field

PhD in Computer Science or related field (preferred)

Tools

LangChain

LlamaIndex

RLHF

Job description

Waymo is seeking a Senior ML Engineer in the Semantics org to develop and deploy state-of-the-art computer vision and multimodal models. You will build an AI agent framework that integrates large foundation models with perception outputs and knowledge bases, and apply RL and fine-tuning to improve captioning and reasoning.

You will collaborate with ML Infra, Perception, Behavior, and AI Foundation teams, leading complex projects in a fast-paced R&D environment, and own the full lifecycle from