Senior ML Engineer: Predictive World Models & Multimodal AI

XPENG; Santa Clara (CA); On-site; USD 175,000 - 296,000; Full time

Job summary

XPENG is seeking Machine Learning Engineers with strong expertise in generative modeling and large-scale deep learning systems.

You will research, implement, and evaluate world models that learn the dynamics of the physical world from large-scale multimodal data — predicting how a scene evolves under an agent's actions, and serving as a learned simulator for training and evaluating driving and robotic policies.

Qualifications

• MS or PhD level education in Engineering or Computer Science with a focus on Deep Learning, Computer Vision, Generative Models, or a related field, or equivalent experience.
• Strong experience in applied deep learning including model architecture design, large-scale model training, data curation, and empirical analysis.
• 1-3 years + of experience working with DL frameworks such as PyTorch, including hands-on experience with distributed training.
• Strong Python programming experience with software design skills.
• Solid understanding of data structures, algorithms, code optimization and large-scale data processing.
• Excellent problem-solving skills, including the ability to design controlled experiments and draw sound conclusions from noisy training signals.

Responsibilities

• Research and develop predictive world models that learn how the physical world evolves, forecasting the future state of a scene from large-scale multimodal driving and robotics data.
• Develop high-quality multi-view future prediction and generation, supporting both action-conditioned rollouts and formulations that forecast the future without explicit action conditioning.
• Work at the boundary between world modeling and policy learning: develop architectures in which a shared backbone both predicts the future and produces trajectories or actions, and apply predictive pre-training to improve Vision-Language-Action (VLA) driving performance.
• Extend prediction beyond 2D pixel into a shared multimodal latent space that spans 3D scene representations such as Gaussian Splatting, together with occupancy and reward signals, so that a single model can support simulation, evaluation, and policy training.
• Advance cross-embodiment generalization: design unified observation and action representations, together with embodiment-conditioning mechanisms, so that a single world model transfers across vehicles, robots, and sensor configurations with only few-shot data.
• Define and build the evaluation methodology for predictive world models, spanning representation quality, prediction accuracy, generation fidelity, physical plausibility, long-horizon rollout consistency, and ultimately closed-loop policy performance, then feed the resulting models back into training as a source of synthetic data and corner-case simulation.

Skills

Deep learning

Computer vision

Generative models

PyTorch

Distributed training

Python

Education

MS/PhD in Engineering or CS

Tools

FSDP

DeepSpeed

Megatron-style parallelism

Job description

XPENG is seeking Machine Learning Engineers with strong expertise in generative modeling and large-scale deep learning systems.

You will research, implement, and evaluate world models that learn the dynamics of the physical world from large-scale multimodal data — predicting how a scene evolves under an agent's actions, and serving as a learned simulator for training and evaluating driving and robotic policies.