Senior Vision-Language ML Engineer, Autonomous Driving

XPENG; Santa Clara (CA); On-site; USD 175,000 - 296,000; Full time

Benefits offered by this job

Massive real-world data

Industry-scale compute

Top-tier researchers and engineers

Competitive compensation

Snacks, meals, and team events

Job summary

XPENG is seeking a Machine Learning Engineer / Research Scientist to drive the development of its Vision-Language-Action foundation model for autonomous driving. You will design and train large-scale multi-modal architectures that fuse vision, language, and control across fleet data and simulations.

You will collaborate with top researchers and engineers to scale training on thousands of GPUs, develop cross-modal alignment methods, and optimize deployment while pursuing state-of-the-art results

Qualifications

• Master’s degree or higher in Computer Science, Electrical/Computer Engineering, or related field, with 3+ years of deep learning research or productization experience.
• Strong proficiency in PyTorch and modern transformer-based model design.
• Experience in large-scale pretraining or multi-modal modeling (vision, language, or planning).
• Deep understanding of representation learning, temporal modeling, and self-supervised or reinforcement learning techniques.

Responsibilities

• Design and implement large-scale multi-modal architectures (e.g., vision-language-action transformers) for end-to-end autonomous driving.
• Develop pretraining and fine-tuning strategies leveraging massive labeled and unlabeled fleet data (images, video, LiDAR, CAN bus, maps, human driving behaviors, etc.).
• Research and integrate cross-modal alignment (e.g., visual grounding, temporal reasoning, policy distillation, imitation and reinforcement learning) to improve model interpretability and action quality.
• Collaborate with infrastructure engineers to scale training across thousands of GPUs using distributed training frameworks (FSDP, DDP, etc.).
• Conduct systematic ablation, evaluation, and visualization of model behavior across perception, reasoning, and planning tasks.
• Contribute to model deployment optimization, including quantization, export, and latency-accuracy trade-offs for onboard execution.

Skills

PyTorch

transformer models

multi-modal modeling

distributed training

vision-language-action

reinforcement learning

Education

Master’s degree or higher in CS/EE

PhD in CS/CE/EE (preferred)

Tools

FSDP

DDP

Job description

XPENG is seeking a Machine Learning Engineer / Research Scientist to drive the development of its Vision-Language-Action foundation model for autonomous driving. You will design and train large-scale multi-modal architectures that fuse vision, language, and control across fleet data and simulations.

You will collaborate with top researchers and engineers to scale training on thousands of GPUs, develop cross-modal alignment methods, and optimize deployment while pursuing state-of-the-art results