Remote GPU Cluster Engineer & Automation Lead

AI Chopping Block; Northern (KY); Hybrid; USD 150,000 - 230,000; Full time

Job summary

ElevenLabs seeks an experienced HPC Infrastructure Engineer to own our GPU cluster infrastructure from provisioning to performance tuning. You will build automation to keep the fleet healthy with minimal human intervention and manage OS images, NVIDIA drivers, CUDA, container runtimes, and NCCL.

You’ll run and optimize Slurm-like job scheduling, design fast storage, and diagnose bottlenecks across hardware and software.

Qualifications

• Experience running large-scale Linux server or GPU environments in production.
• Deep NVIDIA stack knowledge (drivers, CUDA, NCCL, DCGM) or equivalent systems experience.
• Comfort with bare-metal environments, server hardware and high-speed networking.
• Solid automation skills in Python and/or Bash; IaC tooling like Ansible or Terraform.
• Ability to analyze metrics/logs to diagnose issues and improve performance.
• Willingness to own end-to-end scope including datacenter tasks.

Responsibilities

• Operate and optimize GPU clusters from provisioning to capacity planning.
• Develop automation to maintain fleet health with minimal human intervention.
• Manage OS images, NVIDIA drivers, CUDA, container runtimes, and NCCL.
• Run and tune scheduling for researchers to maximize compute efficiency.
• Build high-performance storage for datasets and checkpoints.
• Identify and fix performance bottlenecks across hardware and software.
• Evaluate rented GPU capacity and ensure provider SLAs are met.
• Perform hands-on hardware work when needed (racking, cabling, diagnostics).
• Maintain cluster security through access control and network isolation.

Skills

Linux server administration

NVIDIA stack expertise

Automation with Python/Bash

Metrics & logs analysis (PromQL)

End-to-end ownership

Datacenter hardware handling

Tools

Ansible

Terraform

PXE provisioning

BMC/IPMI/Redfish automation

Job description

ElevenLabs seeks an experienced HPC Infrastructure Engineer to own our GPU cluster infrastructure from provisioning to performance tuning. You will build automation to keep the fleet healthy with minimal human intervention and manage OS images, NVIDIA drivers, CUDA, container runtimes, and NCCL.

You’ll run and optimize Slurm-like job scheduling, design fast storage, and diagnose bottlenecks across hardware and software.