Principal AI Infrastructure Engineer - Highrise AI

LinuxCareers; United States; Remote; USD 170,000 - 250,000; Full time

Job summary

Highrise AI is a GPU-native infrastructure platform serving AI workloads at scale. We seek a Principal AI Infrastructure Engineer to bring large-scale GPU clusters from physical install to production readiness, providing deep expertise in cluster integration, bare-metal provisioning, and performance optimization.

You will lead the bring-up of racked AI clusters, validate BIOS/BMC, drivers, OS, and cluster software, and serve as senior escalation for complex production incidents affecting compute

Qualifications

• Bachelor's degree in computer science, engineering, or related field, or equivalent practical experience.
• 8+ years of hands-on Linux systems engineering, HPC, cloud, or large-scale infrastructure experience, including senior-level troubleshooting.
• Proven experience bringing GPU or HPC clusters into production and qualifying systems against performance and reliability targets.
• Strong expertise in Linux, automation (Python/Bash/Ansible), bare-metal provisioning, configuration management, and systems diagnostics.
• Hands-on experience with NVIDIA GPU platforms, NCCL, Field Diagnostics, and multi-node performance testing.

Responsibilities

• Lead technical bring-up of racked, powered, and cabled AI clusters through production readiness and customer handoff.
• Establish and validate system baselines across BIOS/BMC, firmware, GPU drivers, operating systems, and cluster software stacks.
• Deploy and optimize automated bare-metal provisioning, imaging, node configuration, and lifecycle workflows at scale.
• Execute NVIDIA Field Diagnostics, L11/L12 validation, burn-in, stability, and hardware health qualification procedures.
• Serve as senior escalation point for complex production incidents involving compute nodes, GPU software, provisioning, scheduling, and degraded performance.

Skills

Linux systems engineering

HPC

Troubleshooting

Automation

GPU compute

Education

Bachelor's degree or equivalent

Tools

Python

Bash

Ansible

NVIDIA NCCL

Bare-metal provisioning

Job description

Highrise AI is a GPU-native infrastructure platform serving AI workloads at scale. This role is a Principal AI Infrastructure Engineer responsible for bringing large-scale GPU compute clusters from physical installation through production readiness, providing deep technical expertise in cluster integration, bare-metal provisioning, performance optimization, and Tier 3 operational support.

What You'll Do

• Lead technical bring-up of racked, powered, and cabled AI clusters through production readiness and customer handoff
• Establish and validate system baselines across BIOS/BMC, firmware, GPU drivers, operating systems, and cluster software stacks
• Deploy and optimize automated bare-metal provisioning, imaging, node configuration, and lifecycle workflows at scale
• Execute NVIDIA Field Diagnostics, L11/L12 validation, burn-in, stability, and hardware health qualification procedures
• Serve as senior escalation point for complex production incidents involving compute nodes, GPU software, provisioning, scheduling, and degraded performance

What You Need

• Bachelor's degree in computer science, engineering, or related field, or equivalent practical experience
• 8+ years of hands-on Linux systems engineering, HPC, cloud, or large-scale infrastructure experience, including senior-level troubleshooting
• Proven experience bringing GPU or HPC clusters into production and qualifying systems against performance and reliability targets
• Strong expertise in Linux, automation (Python/Bash/Ansible), bare-metal provisioning, configuration management, and systems diagnostics
• Hands-on experience with NVIDIA GPU platforms, NCCL, Field Diagnostics, and multi-node performance testing

Nice to Have

• Practical experience with Kubernetes and Slurm
• Familiarity with high-speed Ethernet, InfiniBand/RDMA, and parallel or distributed storage