Software Engineer, AI Training Systems (GPU-Scale)

AI Pulse; Palo Alto (CA); On-site; USD 176,000 - 558,000; Full time

Benefits offered by this job

Medical plans

Fertility benefits

Dental & vision plans

HSA contributions

FSA

401(k) & ESPP

Life & AD&D insurance

Disability insurance

Employee assistance program

Paid time off & holidays

Back-up childcare

Voluntary benefits

Weight loss programs

Tesla Babies program

Commuter benefits

Employee discounts

Job summary

Tesla, based in Palo Alto, seeks a Software Engineer for the Autopilot AI Infrastructure team. You will reinforce, optimize, and scale infrastructure supporting AI research for Autopilot and the Optimus project.

You’ll work on tuning data pipelines and GPU-heavy workflows, with emphasis on low-latency, high-throughput training. Strong Python/C++ and CUDA experience, PyTorch familiarity, and a track record of scaling distributed training across multiple GPUs are required.

Qualifications

• Practical experience programming in Python and/or C/C++.
• Experience programming in CUDA, cuDNN or Triton, particularly in AI workloads.
• Experience profiling and optimizing CPU-GPU interactions.
• Experience working with training frameworks (ideally PyTorch).
• Proficient in system-level software, hardware-software interactions and resource utilization.
• Experience with parallel programming concepts and primitives.
• Understanding of modern machine learning concepts and state of the art deep learning.
• Experience scaling neural networks training across many GPUs.

Responsibilities

• Reduce wall clock time to convergence of training jobs by identifying bottlenecks in the ML stack.
• Integrate efficient, low-level code with the high-level training framework.
• Profile workloads and implement solutions to increase training efficiency.
• Optimize workloads for CPU/GPU utilization, data throughput, and networking.

Skills

Python/C++

CUDA/cuDNN/Triton

CPU-GPU profiling

PyTorch

System-level software

Parallel programming

Deep learning concepts

Distributed training

Tools

CUDA

cuDNN

Triton

Job description

Tesla, based in Palo Alto, seeks a Software Engineer for the Autopilot AI Infrastructure team. You will reinforce, optimize, and scale infrastructure supporting AI research for Autopilot and the Optimus project.

You’ll work on tuning data pipelines and GPU-heavy workflows, with emphasis on low-latency, high-throughput training. Strong Python/C++ and CUDA experience, PyTorch familiarity, and a track record of scaling distributed training across multiple GPUs are required.