Senior ML Engineer: Predictive World Models for Autonomy

XPENG; Santa Clara (CA); On-site; USD 175,000 - 296,000; Full time

Benefits offered by this job

Snacks, lunches, dinners, and fun acts

Job summary

XPENG seeks a Machine Learning Engineer specialized in generative modeling and large-scale deep learning systems. You will research, implement, and evaluate world models predicting scene dynamics from multimodal driving and robotics data.

The role involves diffusion/flow-matching architectures, video tokenizers, and transformer-based multimodal backbones, across autonomous fleet data. You will collaborate with experts in CV, generative AI, and AI systems, leveraging XPENG's vast real-world

Qualifications

• MS or PhD in Engineering or Computer Science or equivalent experience.
• Strong experience in applied deep learning including model architecture design, large-scale model training, data curation and empirical analysis.
• 1-3 years+ of experience with DL frameworks such as PyTorch, including distributed training (FSDP, DeepSpeed, or Megatron-style parallelism).
• Strong Python programming experience with software design skills.
• Solid understanding of data structures, algorithms, code optimization and large-scale data processing.
• Excellent problem-solving skills, including designing controlled experiments and drawing sound conclusions from noisy training signals.

Responsibilities

• Research and develop predictive world models that learn how the physical world evolves, forecasting the future state from large-scale multimodal driving and robotics data.
• Develop high-quality multi-view future prediction and generation, supporting action-conditioned rollouts and future forecasting without explicit action conditioning.
• Work at the boundary between world modeling and policy learning: create architectures where a shared backbone predicts the future and produces trajectories or actions, enabling predictive pre-training for Vision-Language-Action driving performance.
• Extend prediction beyond 2D pixels into a shared multimodal latent space that spans 3D scene representations (Gaussian Splatting) with occupancy and reward signals for simulation, evaluation, and policy training.
• Advance cross-embodiment generalization: design unified observation and action representations and embodiment-conditioning to transfer a single world model across vehicles, robots, and sensor configurations.
• Define and build evaluation methodology for predictive world models, including representation quality, prediction accuracy, generation fidelity, and long-horizon rollout consistency, feeding back into training.

Skills

Applied deep learning

PyTorch

Distributed training

Python

Algorithms

Problem solving

Generative models

World models

Multimodal models

Performance optimization

Education

MS or PhD in Engineering or CS

Tools

PyTorch

DeepSpeed

Megatron

Video tokenizers

Job description

XPENG seeks a Machine Learning Engineer specialized in generative modeling and large-scale deep learning systems. You will research, implement, and evaluate world models predicting scene dynamics from multimodal driving and robotics data.

The role involves diffusion/flow-matching architectures, video tokenizers, and transformer-based multimodal backbones, across autonomous fleet data. You will collaborate with experts in CV, generative AI, and AI systems, leveraging XPENG's vast real-world