Lead ML Systems Engineer — Distributed GPU Training & Infra

Nvidia Corporation; Santa Clara (CA); On-site; USD 224,000 - 431,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA in Santa Clara, CA is seeking a Deep Learning Software Infrastructure Engineer to advance autonomous vehicle training infrastructure. You will build and scale libraries to enable end-to-end training on thousands of GPUs and massive datasets, collaborating with research and platform teams.

Qualifications include 12+ years in high-performance distributed systems, deep learning frameworks (PyTorch preferred), and experience with Slurm, Kubernetes, RoCE, and Lustre.

Qualifications

• BS, MS or PhD in Computer Science, Electrical/Computer Engineering, or a related field, or equivalent experience.
• 12+ years of professional experience building and scaling high-performance distributed systems, ideally in ML, HPC, or large-scale data infrastructure.
• Extensive knowledge in deep learning frameworks (PyTorch preferred), large scale training (DDP/FSDP, NCCL, tensor/pipeline parallelism), and performance profiling.

Responsibilities

• Crafting, scaling, and hardening deep learning infrastructure libraries and frameworks for training on multi-thousand GPU clusters.
• Improving efficiency throughout the training stack: data loaders, distributed training, scheduling, and performance monitoring.
• Building robust training pipelines and libraries to handle massive video datasets and enable rapid experimentation.
• Collaborating with researchers, model engineers, and internal platform teams to enhance efficiency, minimize stalls, and improve training availability.
• Owning core infrastructure components such as orchestration libraries, distributed training frameworks, and fault-resilient training systems.
• Partnering with leadership to ensure infrastructure scales with growing GPU capacity and dataset size while maintaining developer efficiency and stability.

Skills

PyTorch

NCCL

DDP/FSDP

Kubernetes

Python

Slurm

RoCE

Lustre

Education

BS/MS/PhD in CS/EE or equivalent

Tools

Slurm

Kubernetes

RoCE

Lustre

Job description

NVIDIA in Santa Clara, CA is seeking a Deep Learning Software Infrastructure Engineer to advance autonomous vehicle training infrastructure. You will build and scale libraries to enable end-to-end training on thousands of GPUs and massive datasets, collaborating with research and platform teams.

Qualifications include 12+ years in high-performance distributed systems, deep learning frameworks (PyTorch preferred), and experience with Slurm, Kubernetes, RoCE, and Lustre.