Senior ML Performance Engineer – Scaled Training

Xpengmotors; Santa Clara (CA); On-site; USD 244,000 - 413,000; Full time

Benefits offered by this job

Competitive compensation package

Snacks, lunches, and dinners

Job summary

XPENG is seeking a senior performance engineering leader to optimize GPU training and inference workloads at scale in a demanding AI and autonomous driving environment. You will own end-to-end performance, work across modeling, infrastructure, data, and robotics teams, and deliver reusable components to accelerate multiple projects.

Ideal candidates have 8+ years in performance engineering, deep systems knowledge, and expert C++/Python skills.

Qualifications

• 8+ years of industry experience in performance engineering.
• Hands-on experience optimizing GPU training or inference workloads in production, multi-node scale.
• Proven track record diagnosing performance problems in real production workloads.
• Deep understanding of computer architecture and high-performance computing.
• Strong C++ and Python programming.

Responsibilities

• Training Performance at Scale: Own step time, scaling efficiency and long-run stability for foundation-model training across thousands of accelerators.
• Cross-Team Technical Leadership: Collaborate with modeling, infrastructure, simulation, data-mining and robotics teams on bottlenecks.
• Reusable by Design: Build modular components that become defaults across models and workloads.
• Observability at Scale: Extend performance tooling and metrics across the stack with low overhead.
• Agentic Automation: Build agents that reproduce jobs on demand and verify parity to catch regressions.

Skills

C++

Python

CUDA

Triton

PyTorch

Data structures & algorithms

Education

Master's or PhD in CS/CE/EE

Tools

CUPTI

Perfetto

Job description

XPENG is seeking a senior performance engineering leader to optimize GPU training and inference workloads at scale in a demanding AI and autonomous driving environment. You will own end-to-end performance, work across modeling, infrastructure, data, and robotics teams, and deliver reusable components to accelerate multiple projects.

Ideal candidates have 8+ years in performance engineering, deep systems knowledge, and expert C++/Python skills.