Senior Machine Learning Engineer – Predictive World Model

Xpeng Inc.; Santa Clara, Northern (CA, KY); On-site; USD 175,000 - 296,000; Full time

Benefits offered by this job

Competitive compensation package

Snacks, lunches, dinners, and fun δρα

Job summary

XPENG is seeking Machine Learning Engineers specializing in generative modeling and large-scale deep learning systems. You will research, implement, and evaluate world models predicting how scenes evolve from multimodal driving and robotics data, using diffusion, flow-matching, video tokenizers, and transformer-based backbones.

You will collaborate with experts in CV, generative AI, and AI systems, training on data from our autonomous fleet and robotics platforms.

Qualifications

• MS or PhD in Engineering or Computer Science, or equivalent experience.
• Strong experience in applied deep learning including model architecture design, large-scale model training, data curation and empirical analysis.
• 1-3 years+ with DL frameworks such as PyTorch and distributed training (FSDP, DeepSpeed, Megatron-style).
• Strong Python programming experience with software design skills.
• Solid understanding of data structures, algorithms, and large-scale data processing.

Responsibilities

• Research and develop predictive world models forecasting future states from multimodal data.
• Develop high-quality multi-view future prediction and generation for action-conditioned rollouts and unconditioned forecasts.
• Design architectures where a shared backbone predicts futures and generates trajectories or actions for policy learning.
• Extend prediction into a 3D multimodal latent space using Gaussian Splatting with occupancy and reward signals.
• Advance cross-embodiment generalization across vehicles and robots with few-shot data.
• Define and build evaluation methodologies for predictive world models and feed results back into training.

Skills

Deep Learning

Python Programming

Distributed Training

Research & Analysis

Multimodal Data

Education

MS or PhD

Tools

PyTorch

DeepSpeed

Megatron

FSDP

Job description

XPENG is a leading smart technology company at the forefront of innovation, integrating advanced AI and autonomous driving technologies into its vehicles, including electric vehicles (EVs), electric vertical take-off and landing (eVTOL) aircraft, and robotics. With a strong focus on intelligent mobility, XPENG is dedicated to reshaping the future of transportation through cutting-edge R&D in AI, machine learning, and smart connectivity.

We are seeking Machine Learning Engineers with strong expertise in generative modeling and large-scale deep learning systems, along with solid software development skills. In this role, you will research, implement, and evaluate world models that learn the dynamics of the physical world from large-scale multimodal data — predicting how a scene evolves under an agent's actions, and serving as a learned simulator for training and evaluating driving and robotic policies. You will work with state-of-the-art generative architectures, including diffusion and flow-matching models, video tokenizers, and transformer-based multimodal backbones. You will collaborate with a world-class team of experts in computer vision, generative AI, and AI systems, powered by vast amounts of real-world multimodal data from our autonomous fleet and robotics platforms.

Job Responsibilities:

Research and develop predictive world models that learn how the physical world evolves, forecasting the future state of a scene from large-scale multimodal driving and robotics data.

Develop high-quality multi-view future prediction and generation, supporting both action-conditioned rollouts and formulations that forecast the future without explicit action conditioning.

Work at the boundary between world modeling and policy learning: develop architectures in which a shared backbone both predicts the future and produces trajectories or actions, and apply predictive pre-training to improve Vision-Language-Action (VLA) driving performance.

Extend prediction beyond 2D pixel into a shared multimodal latent space that spans 3D scene representations such as Gaussian Splatting, together with occupancy and reward signals, so that a single model can support simulation, evaluation, and policy training.

Advance cross-embodiment generalization: design unified observation and action representations, together with embodiment-conditioning mechanisms, so that a single world model transfers across vehicles, robots, and sensor configurations with only few-shot data.

Define and build the evaluation methodology for predictive world models, spanning representation quality, prediction accuracy, generation fidelity, physical plausibility, long-horizon rollout consistency, and ultimately closed-loop policy performance, then feed the resulting models back into training as a source of synthetic data and corner-case simulation.

Minimum Skill Requirements:

MS or PhD level education in Engineering or Computer Science with a focus on Deep Learning, Computer Vision, Generative Models, or a related field, or equivalent experience. Open to recent graduates.

Strong experience in applied deep learning including model architecture design, large-scale model training, data curation, and empirical analysis.

1-3 years + of experience working with DL frameworks such as PyTorch, including hands-on experience with distributed training (FSDP, DeepSpeed, or Megatron-style parallelism).

Strong Python programming experience with software design skills.

Solid understanding of data structures, algorithms, code optimization and large-scale data processing.

Excellent problem-solving skills, including the ability to design controlled experiments and draw sound conclusions from noisy training signals.

Preferred Skill Requirements:

Hands on experience with generative models for video or 3D, such as diffusion, flow matching, autoregressive video prediction, or neural scene representations including NeRF and Gaussian Splatting.

Experience with world models or learned simulators for decision making, including model-based reinforcement learning and Vision-Language-Action (VLA) models.

Experience with multimodal foundation models and video tokenizers or VAEs, including pretraining or adapting large pretrained backbones.

Experience with large-scale training infrastructure and performance optimization, such as mixed precision, torch.compile, kernel-level optimization, and multi-node scaling.

What do we provide:

• A fun, supportive and engaging environment.
• Infrastructures and computational resources to support your work.
• Opportunity to work on cutting edge technologies with the top talents in the field.
• Opportunity to make a significant impact on the transportation revolution by the means of advancing autonomous driving.
• Competitive compensation package.
• Snacks, lunches, dinners, and fun activities.

The base salary range for this full-time position is $174,720 - $295,680, in addition to bonus, equity and benefits. Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training.

We are an Equal Opportunity Employer. It is our policy to provide equal employment opportunities to all qualified persons without regard to race, age, color, sex, sexual orientation, religion, national origin, disability, veteran status or marital status or any other prescribed category set forth in federal or state regulations.