Head of GPU & AI Data Center Operations

Programming.com; United States; On-site; USD 180,000 - 280,000; Full time

Job summary

Programming.com seeks an experienced infrastructure leader to scale large GPU and AI data center operations. You will oversee NVIDIA GPU clusters, HPC environments, and multi-site IT infrastructures, ensuring high availability and performance for AI workloads.

Role requires leading production infrastructure teams, strategic capacity planning, and collaboration with executives and vendors. Strong background in GPU hardware, networking, and cluster management is essential.

Qualifications

• 15+ years in Infrastructure Engineering, Data Center Operations, GPU Cloud, or HPC environments.
• 7+ years of leadership experience managing large-scale production infrastructure operations.
• Direct experience managing production NVIDIA GPU clusters is mandatory.
• Strong knowledge of NVIDIA GPU hardware (H100/H200, DGX/HGX) and NVLink/NVSwitch.
• Experience with InfiniBand, RDMA, high-speed networking, Linux, and GPU hardware troubleshooting.
• Familiarity with Slurm, Kubernetes, NVIDIA DCGM, Prometheus, and Grafana.
• Experience managing multi-site operations, vendors, infrastructure deployments, and technical teams.
• Strong leadership, operational strategy, and executive stakeholder management capabilities.

Responsibilities

• Lead end-to-end operations of large-scale GPU clusters, AI infrastructure, and data center IT environments.
• Manage GPU server deployment, monitoring, troubleshooting, maintenance, and hardware lifecycle.
• Ensure high availability, performance, and reliability of AI training and inference infrastructure.
• Oversee multi-site data center operations, infrastructure expansion, and capacity planning.
• Lead major incident management, RCA, operational governance, and SLA/SLO performance.
• Build and manage 24/7 technical operations teams, engineering managers, and site leaders.
• Drive automation, GPU fleet monitoring, operational efficiency, and infrastructure scaling.
• Collaborate with engineering teams, hardware vendors, and executive leadership.

Skills

Infrastructure leadership

Data center operations

GPU cloud infrastructure

HPC knowledge

Linux systems

Networking (InfiniBand, RDMA)

Incident management

Education

Bachelor's degree in a relevant field

Tools

Kubernetes

Prometheus

Grafana

Slurm

NVIDIA DCGM

NVLink/NVSwitch

Job description

Programming.com seeks an experienced infrastructure leader to scale large GPU and AI data center operations. You will oversee NVIDIA GPU clusters, HPC environments, and multi-site IT infrastructures, ensuring high availability and performance for AI workloads.

Role requires leading production infrastructure teams, strategic capacity planning, and collaboration with executives and vendors. Strong background in GPU hardware, networking, and cluster management is essential.