Site Reliability Engineer – GPU/HPC Infrastructure (Remote – Europe)

Saturn Cloud; Deutschland; Remote; EUR 90,000 - 130,000; Full time

Job summary

Saturn Cloud is seeking a GPU/HPC-focused Site Reliability Engineer to operate and troubleshoot large-scale GPU infrastructure across the EU. The role focuses on diagnosing GPU, driver, CUDA, and kernel interactions in a remote, cloud-based environment.

You will work with NVIDIA GPUs, NVML/nvidia-smi, and Kubernetes-based GPU workloads, coordinating with infrastructure teams to resolve production incidents and improve reliability.

Qualifications

• Hands-on experience with NVIDIA datacenter GPUs and CUDA driver issues.
• Experience diagnosing GPU health, Xid errors, and driver failure modes.
• Familiarity with Kubernetes-based GPU workloads and device plugins.

Responsibilities

• Diagnose and resolve production issues affecting large-scale GPU inference infrastructure.
• Troubleshoot NVIDIA datacenter GPUs, drivers, CUDA compatibility, and GPU container runtimes.
• Investigate GPU health issues, including Xid errors and hardware or driver failure modes.
• Diagnose PCIe topology, NUMA, GPU placement, NVLink/NVSwitch issues.
• Troubleshoot multi-GPU and multi-node workloads.
• Investigate high-performance networking and distributed communication failures.
• Determine origin of production issues across application, GPU, network, or hardware layers.
• Work with Kubernetes GPU Operator and device plugins.

Skills

NVIDIA GPU Admin

CUDA & driver

NVML/nvidia-smi

Kubernetes GPU

GPU Troubleshooting

Multi-node workloads

Tools

Prometheus/Grafana

NVIDIA DCGM

Job description

This is a fully remote role open to candidates across the EU.

About Saturn Cloud

Saturn Cloud builds infrastructure for running AI, machine learning, and data workloads at scale. Our platform helps teams develop, deploy, and operate compute-intensive workloads across modern cloud and GPU infrastructure.

We’re looking for a GPU/HPC-focused Site Reliability Engineer based in the EU to help operate and troubleshoot the large-scale GPU infrastructure supporting Saturn Cloud Token Factory.

What You’ll Do

• Diagnose and resolve production issues affecting large-scale GPU inference infrastructure
• Troubleshoot NVIDIA datacenter GPUs, drivers, CUDA compatibility, and GPU container runtimes
• Investigate GPU health issues, including Xid errors and hardware or driver failure modes
• Diagnose PCIe, NUMA, GPU placement, NVLink, and NVSwitch issues
• Troubleshoot multi-GPU and multi-node workloads
• Investigate high-performance networking and distributed communication failures
• Distinguish application and inference-runtime issues from GPU, fabric, topology, node, driver, or hardware failures
• Work with Kubernetes-based GPU workloads and NVIDIA GPU Operator/device plugins
• Use production observability and GPU metrics to diagnose reliability and performance issues
• Work directly with GPU-cloud and infrastructure providers when incidents require hardware or fabric investigation
• Produce clear technical evidence showing where the failure occurs and what the appropriate infrastructure team needs to investigate

What We’re Looking For

Core Skills

• Hands-on experience using AI coding agents and agentic development tools to accelerate engineering, debugging, automation, and operational workflows
• NVIDIA datacenter GPU administration
• NVIDIA driver installation, upgrades, and troubleshooting
• Strong understanding of CUDA and driver compatibility
• Experience with NVML and nvidia-smi
• GPU health diagnostics, including Xid errors and common hardware/driver failure modes
• Understanding of PCIe topology, NUMA, and GPU placement
• Understanding of NVLink and NVSwitch fundamentals
• Familiarity with Kubernetes GPU Operator and device plugins
• Experience running containerized GPU workloads

High-Performance Networking

You should have strong experience with several of the following:

• InfiniBand
• RDMA
• RoCE
• NCCL
• GPUDirect RDMA
• NCCL testing and distributed workload diagnostics
• Bandwidth and latency troubleshooting

You should be able to determine whether a production issue originates in the application, GPU, network fabric, topology, node, or another underlying infrastructure layer.

Inference Infrastructure

You don’t need to be an ML researcher, but you should understand how modern inference workloads exercise GPU infrastructure.

• vLLM, NVIDIA Dynamo, Triton, or comparable inference runtimes
• Tensor and pipeline parallelism
• Model loading and GPU memory consumption
• KV cache
• Continuous batching
• GPU and memory utilization
• Out-of-memory diagnosis
• Multi-GPU and multi-node inference
• Basic inference performance analysis, including throughput, latency, and GPU saturation

Additional Skills

• Kubernetes troubleshooting sufficient to independently investigate GPU workloads inside a cluster
• Prometheus/Grafana and NVIDIA DCGM metrics
• Bash and Python
• Experience with bare-metal GPU clusters or GPU cloud infrastructure
• Familiarity with B200/B300/H200/H100-class systems is highly desirable

Strong Pluses

• NVIDIA DCGM
• NVIDIA Dynamo
• KAI Scheduler or Grove
• Spectrum-X
• OFED/DOCA
• Slurm or HPC cluster administration
• Kubernetes-based GPU clouds
• Large GPU fleet operations
• GPU burn-in, qualification, and health-check tooling

What Success Looks Like

When an inference workload becomes unavailable or significantly slower, you can determine whether the problem is caused by the inference runtime, GPU memory pressure, a failed GPU, a driver/kernel interaction, PCIe/NVLink/NVSwitch topology, NCCL, RDMA/InfiniBand fabric, or the underlying node.

You can collect enough evidence to distinguish a Saturn Cloud software issue from an operator infrastructure or hardware problem and communicate precisely what an infrastructure provider needs to investigate.

You’re the person the team turns to when Kubernetes looks healthy, but the GPU workload isn’t.

Why Saturn Cloud

• Remote-first culture with a high-trust, high-ownership environment
• Work directly with cutting-edge GPU and AI infrastructure
• Solve complex production problems spanning hardware, networking, Kubernetes, and modern inference systems
• Help shape the reliability and operational practices behind large-scale AI workloads