Senior AI Compute Scheduler: Scale Large GPU Clusters

2100 NVIDIA USA; Santa Clara (CA); On-site; USD 184,000 - 357,000; Full time

Job summary

NVIDIA is seeking a seasoned software engineer to design and build scheduling for large GPU fleets across Slurm and Kubernetes. You will implement queueing, topology-aware placement on NVLink systems, preemption, reservations, and backfill across deep queues.

You will also design policy layers for fair share and quotas, run Slurm as a cloud-native service on Kubernetes, and extend Kueue for training, RL, sandboxes and inference.

Qualifications

• BS or MS in Computer Science, Computer Engineering, Electrical Engineering or related field, or equivalent experience.
• 6+ years building production systems software.
• Deep experience with at least one batch or cluster scheduler (Slurm, Kueue, Volcano, KAI, YuniKorn or similar).
• Strong programming in Go, C or C++, plus Python for tooling and analysis.
• Hands-on Kubernetes experience: controllers/operators, CRDs, scheduling and resource management.
• Solid Linux systems knowledge: containers (Docker, Podman, Enroot or Apptainer), cgroups, networking basics and debugging under load.
• Track record of owning a service in production and improving it.
• Clear written and spoken communication with engineers and researchers.

Responsibilities

• Design and build scheduling for large GPU fleets on Slurm and Kubernetes.
• Implement queueing, topology-aware placement on rack-scale NVLink systems, preemption, reservations and backfill for deep queues.
• Create the policy layer: fair share, quotas and allocations across many research teams.
• Run Slurm as a cloud-native service on Kubernetes; extend Kubernetes batch scheduling (Kueue) for training, RL, sandboxes and inference.
• Build services around the scheduler: reservation automation, node-health-aware placement, and cross-cluster workflows.

Skills

Go

C/C++

Python

Kubernetes

Linux

Education

BS/MS in CS/CE or equivalent

Tools

Kubernetes

Docker

Kueue

Slurm

Job description

NVIDIA is seeking a seasoned software engineer to design and build scheduling for large GPU fleets across Slurm and Kubernetes. You will implement queueing, topology-aware placement on NVLink systems, preemption, reservations, and backfill across deep queues.

You will also design policy layers for fair share and quotas, run Slurm as a cloud-native service on Kubernetes, and extend Kueue for training, RL, sandboxes and inference.