Senior Machine Learning Engineer – Predictive World Model

Socket.dev; Santa Clara (CA); On-site; USD 175,000 - 296,000; Full time

Benefits offered by this job

Competitive compensation

Snacks, lunches, dinners

Career growth opportunities

Job summary

XPENG is seeking Machine Learning Engineers specializing in generative modeling and large‑scale deep learning systems. You will research, implement, and evaluate world models that learn dynamics from large multimodal data, serving as learned simulators for training driving and robotics policies.

Work with diffusion and flow‑matching models, video tokenizers, and transformer‑based backbones. You will collaborate with a world‑class team of experts in computer vision, generative AI, and AI systems,

Qualifications

• MS or PhD level education in Engineering or Computer Science with a focus on Deep Learning or related field.

Responsibilities

• Research and develop predictive world models that forecast scene evolution from multimodal driving and robotics data.
• Develop high‑quality multi‑view future prediction supporting action‑conditioned and unconditional rollouts.
• Extend prediction into a 3D multimodal latent space and enable single model for simulation, evaluation and policy training.

Skills

Deep Learning

Python

Experiment design

Algorithms

PyTorch

Distributed training

Education

MS/PhD in Engineering or CS

Tools

PyTorch

DeepSpeed

Megatron

Job description

XPENG is a leading smart technology company at the forefront of innovation, integrating advanced AI and autonomous driving technologies into its vehicles, including electric vehicles (EVs), electric vertical take-off and landing (eVTOL) aircraft, and robotics. With a strong focus on intelligent mobility, XPENG is dedicated to reshaping the future of transportation through cutting‑edge R&D in AI, machine learning, and smart connectivity.

We are seeking Machine Learning Engineers with strong expertise in generative modeling and large‑scale deep learning systems, along with solid software development skills. In this role, you will research, implement, and evaluate world models that learn the dynamics of the physical world from large‑scale multimodal data — predicting how a scene evolves under an agent's actions, and serving as a learned simulator for training and evaluating driving and robotic policies. You will work with state‑of‑the‑art generative architectures, including diffusion and flow‑matching models, video tokenizers, and transformer‑based multimodal backbones. You will collaborate with a world‑class team of experts in computer vision, generative AI, and AI systems, powered by vast amounts of real‑world multimodal data from our autonomous fleet and robotics platforms.

Job Responsibilities:

• Research and develop predictive world models that learn how the physical world evolves, forecasting the future state of a scene from large‑scale multimodal driving and robotics data.
• Develop high‑quality multi‑view future prediction and generation, supporting both action‑conditioned rollouts and formulations that forecast the future without explicit action conditioning.
• Work at the boundary between world modeling and policy learning: develop architectures in which a shared backbone both predicts the future and produces trajectories or actions, and apply predictive pre‑training to improve Vision‑Language‑Action (VLA) driving performance.
• Extend prediction beyond 2D pixel into a shared multimodal latent space that spans 3D scene representations such as Gaussian Splatting, together with occupancy and reward signals, so that a single model can support simulation, evaluation, and policy training.
• Advance cross‑embodiment generalization: design unified observation and action representations, together with embodiment‑conditioning mechanisms, so that a single world model transfers across vehicles, robots, and sensor configurations with only few‑shot data.
• Define and build the evaluation methodology for predictive world models, spanning representation quality, prediction accuracy, generation fidelity, physical plausibility, long‑horizon rollout consistency, and ultimately closed‑loop policy performance, then feed the resulting models back into training as a source of synthetic data and corner‑case simulation.

Minimum Skill Requirements:

• MS or PhD level education in Engineering or Computer Science with a focus on Deep Learning, Computer Vision, Generative Models, or a related field, or equivalent experience. Open to recent graduates.
• Strong experience in applied deep learning including model architecture design, large‑scale model training, data curation, and empirical analysis.
• 1‑3 years + of experience working with DL frameworks such as PyTorch, including hands‑on experience with distributed training (FSDP, DeepSpeed, or Megatron‑style parallelism).
• Strong Python programming experience with software design skills.
• Solid understanding of data structures, algorithms, code optimization and large‑scale data processing.
• Excellent problem‑solving skills, including the ability to design controlled experiments and draw sound conclusions from noisy training signals.

Preferred Skill Requirements:

• Hands on experience with generative models for video or 3D, such as diffusion, flow matching, autoregressive video prediction, or neural scene representations including NeRF and Gaussian Splatting.
• Experience with world models or learned simulators for decision making, including model‑based reinforcement learning and Vision‑Language‑Action (VLA) models.
• Experience with multimodal foundation models and video tokenizers or VAEs, including pretraining or adapting large pretrained backbones.
• Experience with large‑scale training infrastructure and performance optimization, such as mixed precision, torch.compile, kernel‑level optimization, and multi‑node scaling.

What do we provide:

• A fun, supportive and engaging environment.
• Infrastructures and computational resources to support your work.
• Opportunity to work on cutting edge technologies with the top talents in the field.
• Opportunity to make a significant impact on the transportation revolution by the means of advancing autonomous driving.
• Competitive compensation package.
• Snacks, lunches, dinners, and fun activities.

The base salary range for this full‑time position is $174,720 - $295,680, in addition to bonus, equity and benefits. Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by work location and additional factors, including job‑related skills, experience, and relevant education or training.

We are an Equal Opportunity Employer. It is our policy to provide equal employment opportunities to all qualified persons without regard to race, age, color, sex, sexual orientation, religion, national origin, disability, veteran status or marital status or any other prescribed category set forth in federal or state regulations.