Head of Data Center Operations – GPU & AI Infrastructure

Programming.com; United States; On-site; USD 180,000 - 280,000; Full time

Job summary

Programming.com seeks an experienced infrastructure leader to scale large GPU and AI data center operations. You will oversee NVIDIA GPU clusters, HPC environments, and multi-site IT infrastructures, ensuring high availability and performance for AI workloads.

Role requires leading production infrastructure teams, strategic capacity planning, and collaboration with executives and vendors. Strong background in GPU hardware, networking, and cluster management is essential.

Qualifications

• 15+ years in Infrastructure Engineering, Data Center Operations, GPU Cloud, or HPC environments.
• 7+ years of leadership experience managing large-scale production infrastructure operations.
• Direct experience managing production NVIDIA GPU clusters is mandatory.
• Strong knowledge of NVIDIA GPU hardware (H100/H200, DGX/HGX) and NVLink/NVSwitch.
• Experience with InfiniBand, RDMA, high-speed networking, Linux, and GPU hardware troubleshooting.
• Familiarity with Slurm, Kubernetes, NVIDIA DCGM, Prometheus, and Grafana.
• Experience managing multi-site operations, vendors, infrastructure deployments, and technical teams.
• Strong leadership, operational strategy, and executive stakeholder management capabilities.

Responsibilities

• Lead end-to-end operations of large-scale GPU clusters, AI infrastructure, and data center IT environments.
• Manage GPU server deployment, monitoring, troubleshooting, maintenance, and hardware lifecycle.
• Ensure high availability, performance, and reliability of AI training and inference infrastructure.
• Oversee multi-site data center operations, infrastructure expansion, and capacity planning.
• Lead major incident management, RCA, operational governance, and SLA/SLO performance.
• Build and manage 24/7 technical operations teams, engineering managers, and site leaders.
• Drive automation, GPU fleet monitoring, operational efficiency, and infrastructure scaling.
• Collaborate with engineering teams, hardware vendors, and executive leadership.

Skills

Infrastructure leadership

Data center operations

GPU cloud infrastructure

HPC knowledge

Linux systems

Networking (InfiniBand, RDMA)

Incident management

Education

Bachelor's degree in a relevant field

Tools

Kubernetes

Prometheus

Grafana

Slurm

NVIDIA DCGM

NVLink/NVSwitch

Job description

We are looking for an experienced infrastructure leader to manage and scale large-scale GPU and AI data center operations. The role requires strong technical expertise in NVIDIA GPU clusters, high-performance computing (HPC), infrastructure reliability, and leadership of mission-critical operations teams.

Key Responsibilities

• Lead end-to-end operations of large-scale GPU clusters, AI infrastructure, and data center IT environments.
• Manage GPU server deployment, monitoring, troubleshooting, maintenance, and hardware lifecycle.
• Ensure high availability, performance, and reliability of AI training and inference infrastructure.
• Oversee multi-site data center operations, infrastructure expansion, and capacity planning.
• Lead major incident management, RCA, operational governance, and SLA/SLO performance.
• Build and manage 24/7 technical operations teams, engineering managers, and site leaders.
• Drive automation, GPU fleet monitoring, operational efficiency, and infrastructure scaling.
• Collaborate with engineering teams, hardware vendors, and executive leadership.

Required Skills

• 15+ years in Infrastructure Engineering, Data Center Operations, GPU Cloud, or HPC environments.
• 7+ years of leadership experience managing large-scale production infrastructure operations.
• Direct experience managing production NVIDIA GPU clusters is mandatory.
• Strong knowledge of NVIDIA H100/H200/B200/GB200, DGX/HGX, NVLink, and NVSwitch or comparable GPU infrastructure.
• Experience with InfiniBand, RDMA, high-speed networking, Linux, and GPU hardware troubleshooting.
• Familiarity with Slurm, Kubernetes, NVIDIA DCGM, Prometheus, and Grafana.
• Experience managing multi-site operations, vendors, infrastructure deployments, and technical teams.
• Strong leadership, operational strategy, and executive stakeholder management capabilities.

Preferred

• Experience managing thousands of GPUs, hyperscale AI infrastructure, NVIDIA DGX SuperPOD, or GPU cloud platforms.

Important: This is a technical GPU infrastructure leadership position, not a traditional facilities, HVAC, or electrical operations role.