Customer Reliability Engineer - AI Infrastructure

Hamilton Barnes; San Francisco (CA); Remote; USD 67,000 - 101,000; Full time

Benefits offered by this job

Meaningful equity

Comprehensive healthcare

Unlimited PTO

Job summary

Hamilton Barnes is hiring a Support Engineer for a fully remote EU-based role. You will own customer issues end-to-end, debugging Linux and Kubernetes, diagnosing GPU failures, and building automation to prevent recurrence.

The ideal candidate brings deep Linux troubleshooting, Kubernetes knowledge, Python or Bash automation, and excellent written communication. Familiarity with NVIDIA GPUs and monitoring tools is a strong plus.

Qualifications

• Strong hands on Linux troubleshooting from the command line across logs, processes, networking and disk.
• Working knowledge of Kubernetes including pods, nodes, deployments, services and scheduling, and how to investigate when they break.
• Proficiency in Python or Bash for automating repetitive operational tasks.
• Excellent written communication, with the ability to explain technical problems clearly to frustrated customers.
• Sound judgement under pressure, knowing what to check first, when to escape and how to keep stakeholders informed.
• Willingness to join a 24/7 on call rotation, based in the EU and working fully remote.
• Experience with NVIDIA GPUs in production (CUDA, DCGM), InfiniBand/RoCE or NCCL, Slurm, Prometheus/Grafana, Terraform/Ansible/Helm, or a previous customer facing technical role is a strong plus.

Responsibilities

• Own customer issues end to end, debugging at the Linux and Kubernetes layer across processes, networking, storage, scheduling and kernel logs.
• Diagnose GPU failures including driver and device plugin issues, XID errors, thermal throttling and multi node job failures.
• Take part in a 24/7 on call rotation, leading first response on alerts and outages and keeping customers clearly updated throughout.
• Determine whether faults sit with the provider, the platform or the customer, then open and chase tickets with compute providers to resolution.
• Build Python and Bash automation alongside monitoring for cluster health, GPU telemetry, dashboards and alerting.
• Write post incident reviews and turn them into runbooks, alerts and fixes, keeping customer documentation current.

Skills

Linux troubleshooting

Kubernetes

Python

Bash

Communication

On-call rotation

Tools

CUDA

DCGM

InfiniBand

NCCL

Slurm

Prometheus Grafana

Terraform Ansible Helm

Job description

Join a fast-growing AI infrastructure company building the marketplace and platform through which the world’s fragmented GPU supply is bought, sold, and operated. It connects model builders, research labs, and inference providers with reliable capacity from neoclouds, hyperscalers, and independent data centres worldwide.

The organization is expanding its team and is looking to appoint a Support Engineer for a fully remote, EU-based role. The ideal candidate will be the engineer customers hear from when a node goes dark or a training job fails mid-run, owning each issue from first triage through to resolution and building tooling to stop it recurring. Prior GPU or HPC experience is not required.

Responsibilities:

• Own customer issues end to end, debugging at the Linux and Kubernetes layer across processes, networking, storage, scheduling and kernel logs.
• Diagnose GPU failures including driver and device plugin issues, XID errors, thermal throttling and multi node job failures.
• Take part in a 24/7 on call rotation, leading first response on alerts and outages and keeping customers clearly updated throughout.
• Determine whether faults sit with the provider, the platform or the customer, then open and chase tickets with compute providers to resolution.
• Build Python and Bash automation alongside monitoring for cluster health, GPU telemetry, dashboards and alerting.
• Write post incident reviews and turn them into runbooks, alerts and fixes, keeping customer documentation current.

Skills/Must Have:

• Strong hands on Linux troubleshooting from the command line across logs, processes, networking and disk.
• Working knowledge of Kubernetes including pods, nodes, deployments, services and scheduling, and how to investigate when they break.
• Proficiency in Python or Bash for automating repetitive operational tasks.
• Excellent written communication, with the ability to explain technical problems clearly to frustrated customers.
• Sound judgement under pressure, knowing what to check first, when to escape and how to keep stakeholders informed.
• Willingness to join a 24/7 on call rotation, based in the EU and working fully remote.
• Experience with NVIDIA GPUs in production (CUDA, DCGM), InfiniBand/RoCE or NCCL, Slurm, Prometheus/Grafana, Terraform/Ansible/Helm, or a previous customer facing technical role is a strong plus.

Benefits:

• Meaningful equity
• Comprehensive healthcare
• Unlimited PTO

Salary:

• Competitive salary