Senior HPC GPU Cluster Lead for Deep Learning Infra

NVIDIA; Germany (OH); On-site; USD 58,306 - 101,063; Full time

Job summary

NVIDIA's DLFW Infrastructure team seeks a Senior HPC Cluster Administrator to lead design, deployment, and reliability of our large-scale GPU compute clusters, from DGX/HGX to Grace systems. You will drive architecture across compute, networking, and storage and partner with software and research teams to keep our infrastructure ahead of workloads.

You will own full lifecycle management, implement IaC pipelines, optimize Slurm scheduling, and mentor engineers while advancing observability with

Qualifications

• BS/MS in CS, EE, CE, or equivalent hands-on experience.
• 5+ years of experience deploying and administering large-scale HPC or ML training clusters.
• Deep expertise in Linux systems administration at scale.
• Strong scripting and automation skills in Python and/or bash.
• Hands-on experience with Slurm (scheduling, accounting, cgroup configuration).
• Proficiency with configuration management and IaC (Ansible required; Terraform a plus).
• Experience with container technologies (Docker, Apptainer/Singularity, Kubernetes).
• Solid understanding of high-speed networking (InfiniBand, RoCE, RDMA, EFA).
• Experience with distributed/parallel filesystems and storage architecture.
• Ability to own problems end-to-end and communicate clearly with engineering and management stakeholders.

Responsibilities

• Own the full lifecycle of GPU compute clusters - procurement, provisioning, configuration management, monitoring, and deprecation - across heterogeneous Linux environments.
• Design and scale storage solutions with a clear roadmap for capacity and performance growth.
• Lead automation of infrastructure using IaC tools (Ansible, Terraform) and CI/CD pipelines (GitLab).
• Manage and optimize job scheduling via Slurm, including fair-share policies and partitioning strategies.
• Maintain and improve observability stacks (Prometheus, Grafana, DCGM) and drive proactive resolution of incidents.
• Collaborate with ML engineers and software teams to tune cluster configuration for large-scale distributed training workloads.
• Evaluate and introduce new technologies - networking fabrics (InfiniBand, NVLink, EFA/RDMA), storage tiers, container runtimes - to improve performance and reliability.
• Mentor junior engineers and contribute to team-wide engineering standards.

Skills

Linux admin

Python scripting

Slurm expertise

IaC automation

Docker/Kubernetes

Networking InfiniBand

Education

BS/MS in CS, EE, CE or equivalent

Tools

Ansible

Terraform

Docker

Kubernetes

Prometheus

Grafana

DCGM

nvidia-smi

Job description

NVIDIA's DLFW Infrastructure team seeks a Senior HPC Cluster Administrator to lead design, deployment, and reliability of our large-scale GPU compute clusters, from DGX/HGX to Grace systems. You will drive architecture across compute, networking, and storage and partner with software and research teams to keep our infrastructure ahead of workloads.

You will own full lifecycle management, implement IaC pipelines, optimize Slurm scheduling, and mentor engineers while advancing observability with