Senior AI Training Infrastructure Architect (Multi-GPU)

Nvidia Corporation in; Santa Clara (CA); On-site; USD 224,000 - 431,000; Full time

Job summary

NVIDIA in Santa Clara, CA is seeking a Senior Deep Learning Software Infrastructure Engineer to accelerate autonomous vehicles research by building scalable training libraries and infrastructure for thousands of GPUs and massive datasets.

You will collaborate with researchers and platform teams to improve training efficiency, resilience, and availability, owning core components like orchestration libraries and distributed frameworks.

Qualifications

• 12+ years of professional experience building and scaling high-performance distributed systems, ideally in ML, HPC, or large-scale data infrastructure.
• Extensive knowledge in deep learning frameworks (PyTorch preferred), and large-scale training (DDP/FSDP, NCCL, tensor/pipeline parallelism).
• Strong systems background: datacenter networking, parallel filesystems, storage, and schedulers.
• Proficiency in Python with production-grade libraries and orchestration/automation tools.
• Ability to work closely with multi-functional teams and translate requirements into robust systems.

Responsibilities

• Craft, scale, and harden deep learning infrastructure libraries for training on multi-thousand GPU clusters.
• Improve efficiency across the training stack: data loaders, distributed training, scheduling, and performance monitoring.
• Build robust training pipelines to handle massive datasets and enable rapid experimentation.
• Collaborate with researchers and platform teams to enhance efficiency and training availability.
• Own core infrastructure components such as orchestration libraries and distributed training frameworks.
• Partner with leadership to ensure scalability and developer productivity.

Skills

Distributed systems design

Research collaboration

Production-grade Python

Performance profiling

Education

BS/MS/PhD in CS or related field

Tools

PyTorch

NCCL

DDP/FSDP

RoCE

Lustre

Kubernetes

Slurm

Job description

NVIDIA in Santa Clara, CA is seeking a Senior Deep Learning Software Infrastructure Engineer to accelerate autonomous vehicles research by building scalable training libraries and infrastructure for thousands of GPUs and massive datasets.

You will collaborate with researchers and platform teams to improve training efficiency, resilience, and availability, owning core components like orchestration libraries and distributed frameworks.