Senior GPU Infra Engineer for Distributed AI

Andromeda Cluster; San Francisco (CA); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Andromeda Cluster is seeking a Senior Site Reliability Engineer to design, operate and debug large-scale GPU infrastructure used for distributed training and inference, working directly with customers pushing the limits of modern AI systems.

You will own GPU cluster architecture, optimize performance, ensure reliability, and build automation and observability tooling. The role requires hands-on experience with GPU hardware, Linux internals, Kubernetes and high-speed interconnects.

Qualifications

• Hands-on experience operating large-scale GPU clusters.
• Deep knowledge of GPU memory hierarchies, ECC behavior, and failure modes.
• Production networking with InfiniBand, RoCE, or NVLink in distributed training.
• Proficiency with Linux kernel tuning, drivers, and performance profiling.
• Experience with Kubernetes for GPU workloads and device plugins.

Responsibilities

• GPU Cluster Architecture: design multi-provider, multi-region GPU clusters for large-scale training.
• Customer Technical Partnership: onboard and troubleshoot customers running large-scale training workloads.
• Reliability & Performance Engineering: define SLOs and budgets for GPU infrastructure.
• Networking & Fabric Health: diagnose and resolve high-speed interconnect issues.
• Observability: build visibility into GPU utilization and training performance.
• Automation & Tooling: build automation for provisioning, health checks, and scheduling.
• Incident Leadership: lead postmortems and drive systemic fixes.

Skills

GPU Systems Expertise

High-Performance Networking

Distributed Training & ML Frameworks

Linux & Systems Internals

Kubernetes & Orchestration

Automation & Software Engineering

Observability & Monitoring

Incident Management

Tools

NVIDIA DCGM

nvidia-smi monitoring

Job description

Andromeda Cluster is seeking a Senior Site Reliability Engineer to design, operate and debug large-scale GPU infrastructure used for distributed training and inference, working directly with customers pushing the limits of modern AI systems.

You will own GPU cluster architecture, optimize performance, ensure reliability, and build automation and observability tooling. The role requires hands-on experience with GPU hardware, Linux internals, Kubernetes and high-speed interconnects.