Senior Deep Learning Training Infrastructure Architect

NVIDIA Corporation; California (MO); On-site; USD 224,000 - 357,000; Full time

Job summary

NVIDIA Corporation seeks a Deep Learning Software Infrastructure Engineer to scale training libraries for multi-thousand GPU clusters and to accelerate autonomous driving model development. You will collaborate with researchers and platform teams to improve training efficiency, data handling, and system reliability.

The role emphasizes building robust pipelines and orchestration layers, with ownership of core infrastructure components and emphasis on fault resilience and large-scale

Qualifications

• BS, MS, or PhD in Computer Science, Electrical/Computer Engineering, or related field, or equivalent experience.
• 12+ years of professional experience building and scaling high-performance distributed systems, ideally in ML, HPC, or large-scale data infrastructure.
• Extensive knowledge in deep learning frameworks (PyTorch preferred), large scale training (DDP/FSDP, NCCL), and performance profiling.
• Strong systems background: datacenter networking (RoCE, IB), parallel filesystems (Lustre), storage systems, schedulers (Slurm, Kubernetes).
• Proficiency in Python with production-grade libraries, orchestration layers, and automation tools.

Responsibilities

• Crafting, scaling, and hardening deep learning infrastructure libraries and frameworks for training on multi-thousand GPU clusters.
• Improving efficiency throughout the training stack: data loaders, distributed training, scheduling, and performance monitoring.
• Building robust training pipelines and libraries to handle massive video datasets and enable rapid experimentation.
• Collaborating with researchers, model engineers, and internal platform teams to enhance efficiency and training availability.
• Owning core infrastructure components such as orchestration libraries, distributed training frameworks, and fault-resilient training systems.
• Partnering with leadership to ensure infrastructure scales with growing GPU capacity and dataset size while maintaining developer efficiency and stability.

Skills

Distributed systems

Python

Collaboration

ML collaboration

Leadership

Education

BS, MS, or PhD in CS/EE or related

Tools

PyTorch

NCCL

Slurm

Kubernetes

RoCE

Job description

NVIDIA Corporation seeks a Deep Learning Software Infrastructure Engineer to scale training libraries for multi-thousand GPU clusters and to accelerate autonomous driving model development. You will collaborate with researchers and platform teams to improve training efficiency, data handling, and system reliability.

The role emphasizes building robust pipelines and orchestration layers, with ownership of core infrastructure components and emphasis on fault resilience and large-scale