Principal AI Infrastructure Engineer - Highrise AI

Highrise AI; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Highrise AI is seeking a senior infrastructure lead to bring up GPU-heavy AI clusters from initial rack to customer handoff, ensuring reliable production readiness.

You will establish baselines across BIOS/BMC, firmware, GPU drivers, and OS, deploy scalable bare-metal provisioning, run diagnostics including NCCL tests, and serve as escalation for complex incidents while driving root-cause actions.

Qualifications

• 8+ years of hands-on Linux, HPC, cloud, or large-scale infrastructure experience.
• Proven experience bringing GPU or HPC clusters into production.
• Strong expertise in Linux, Python/Bash/Ansible, and systems diagnostics.
• Practical experience with Kubernetes and Slurm.

Responsibilities

• Lead technical bring-up of racked, powered, cabled AI clusters through customer handoff
• Establish and validate system baselines across BIOS/BMC, firmware, GPU drivers, and OS stacks
• Deploy and optimize automated bare-metal provisioning and lifecycle workflows at scale
• Execute NVIDIA Field Diagnostics, burn-in, and hardware health qualification procedures
• Run and analyze NCCL tests and distributed workload performance benchmarks
• Serve as senior escalation point for complex production incidents
• Perform deep root cause analysis and lead corrective actions for systemic issues

Skills

Linux

HPC

Cloud

Large-scale infra

Python

Bash

Ansible

System diagnostics

Kubernetes

Slurm

Education

Bachelor's degree

Tools

NVIDIA Diagnostics

NCCL tests

Job description

Responsibilities

• Lead technical bring-up of racked, powered, cabled AI clusters through customer handoff
• Establish and validate system baselines across BIOS/BMC, firmware, GPU drivers, and OS stacks
• Deploy and optimize automated bare-metal provisioning and lifecycle workflows at scale
• Execute NVIDIA Field Diagnostics, burn-in, and hardware health qualification procedures
• Run and analyze NCCL tests and distributed workload performance benchmarks
• Serve as senior escalation point for complex production incidents
• Perform deep root cause analysis and lead corrective actions for systemic issues

Requirements

• Bachelor's degree in computer science, engineering, or equivalent practical experience
• 8+ years of hands-on Linux, HPC, cloud, or large-scale infrastructure experience
• Proven experience bringing GPU or HPC clusters into production
• Strong expertise in Linux, Python/Bash/Ansible, and systems diagnostics
• Practical experience with Kubernetes and Slurm

Preferred Qualifications

• Familiarity with high-speed Ethernet, InfiniBand/RDMA, and parallel or distributed storage

About the Company

Highrise AI is a GPU-native infrastructure platform delivering scalable compute across dedicated clusters, managed cloud, and confidential environments. It is a subsidiary of Hut 8 Corp.