Senior / Staff Machine Learning Engineer - Behavior Models for Road Users

Front Door Defense; Foster City (CA); On-site; USD 254,000 - 366,000; Full time

Job summary

Zoox is seeking a Senior / Staff Machine Learning Engineer to develop and deploy behavior models for road users within the simulation stack in Foster City, CA. The role focuses on learning driving agent behaviors using imitation and reinforcement learning, enabling realistic, controllable agents for testing the autonomous driving system.

You will work on large-scale ML pipelines, transformer architectures, and generative models (diffusion/autoregressive) to predict and plan ego-vehicle actions,

Qualifications

• PhD in CS or related field with 4+ years of relevant experience, or a master's with 7+ years.
• Experience in planning, prediction, reinforcement learning, imitation learning, or generative modeling (diffusion, autoregressive models).
• Experience with training and deploying transformer-based model architectures.
• Experience with production ML pipelines: dataset creation, training frameworks, metrics pipelines.
• Fluency in Python ML frameworks and a basic understanding of C++.

Responsibilities

• Develop new deep learning models that use imitation learning and reinforcement learning to generate driving plans for human-like driving agents.
• Estimate the quality of driving plans along safety, progress, comfort and realism.
• Build generative behavior models (autoregressive, diffusion) conditioned on scenario intent for targeted stress-testing.
• Leverage compute, infrastructure and data to push the field forward.
• Develop metrics and tools to analyze errors and understand improvements.
• Collaborate with Perception, Planning, Simulation, and Validation teams to solve autonomous driving challenges.

Skills

Python

Transformer models

Reinforcement Learning

Imitation Learning

Diffusion models

ML pipelines

Education

PhD in CS or related field

Master's degree in CS or related field

Tools

C++

JAX

Job description

Senior / Staff Machine Learning Engineer - Behavior Models for Road Users

Develop and deploy machine learning behavior models for road users in Zoox's simulation stack

Location: Foster City, California

Compensation: $254,000 - 366,000 USD / year

About The Role

ML Agents Machine Learning Engineer

This role in the Offline Driving Intelligence team is responsible for developing/learning behavior models for road users such as cars, bicycles, and pedestrians. These agents populate Zoox's simulations and must be indistinguishable from real road users, yet fully controllable: promptable into the rare, adversarial, safety-critical behaviors we need to test against. This means the team's models directly impact how fast Zoox can train, validate and ship its driving stack. Our team collaborates closely with Planner, Simulation and Validation teams to develop and validate our driving performance. As an ML Agents Machine Learning Engineer, you will work on the bleeding edge of the industry, developing novel machine learning pipelines and models to predict the behavior of other agents in the world and planning the best course of action for the ego vehicle.

In this role, you will:

• Develop new deep learning models that use imitation learning and reinforcement learning to generate driving plans for human-like driving agents.
• Work on novel techniques to estimate the quality of those driving plans along the dimensions of safety, progress, comfort and realism.
• Build generative behavior models (e.g. autoregressive, diffusion) that are conditionable on scenario intent — 'cut off the ego vehicle,' 'jaywalk here' — for targeted stress-testing.
• Leverage our compute, infrastructure and large corpus of data to push boundaries of the field.
• Develop metrics and tools to analyze errors and understand improvements of our systems.
• Collaborate with engineers on Perception, Planning, Simulation, and Validation to solve the overall Autonomous Driving problem.

Qualifications

• PhD degree in computer science or related field and 4+ years of relevant professional experience or master's degree and 7+ years of relevant professional experience
• Experience in one of the following: Planning, Prediction, Reinforcement Learning, Imitation Learning, generative modeling (diffusion, autoregressive models)
• Experience with training and deploying transformer-based model architectures
• Experience with production Machine Learning pipelines: dataset creation, training frameworks, metrics pipelines
• Fluency in Python ML frameworks and a basic understanding of C++

Bonus Qualifications

• Top tier publications (NeurIPS, ICML, CVPR)
• Experience with JAX

$254,000 - $366,000 a year

There are three major components to compensation for this position: salary, Amazon Restricted Stock Units (RSUs), and Zoox Stock Appreciation Rights. A sign-on bonus may be offered as part of the compensation package. The listed range applies only to the base salary. Compensation will vary based on geographic location and level. Leveling, as well as positioning within a level, is determined by a range of factors, including, but not limited to, a candidate's relevant years of experience, domain knowledge, and interview performance. Zoox also offers a comprehensive package of benefits, including paid time off (e.g. sick leave, vacation, bereavement), unpaid time off, Zoox Stock Appreciation Rights, Amazon RSUs, health insurance, long-term care insurance, long-term and short-term disability insurance, and life insurance.