Senior GPU Cloud Reliability Architect for HPC & Kubernetes

NVIDIA Corporation; La Boissière-des-Landes; On-site; EUR 110,000 - 150,000; Full time

Job summary

NVIDIA Corporation is seeking a Day 2 Site Reliability Engineer to join the EMEA NVIDIA Cloud Partner Operations team. You will own fleet-scale production reliability for multi-thousand-GPU deployments, defining MTBI, improving goodput and managing the error budget across partner estates.

The role emphasizes hands-on reliability, SLOs, and automation across open stack Kubernetes, KubeVirt, Slurm, and GPU tooling. Collaboration with NVEX, TAMs, and partner teams is key.

Qualifications

• BS/MS/PhD in Computer Science, Electrical/Computer Engineering, Physics, Mathematics, or related fields, or equivalent experience.
• 8+ years in SRE, production engineering or large-scale infrastructure operations roles.
• Reliability Engineering Practice: ownership of production reliability at scale—SLO and error-budget definition, on-call and incident command, blameless post-incident review, and measurable improvement of availability.
• GPU & HPC Fleet Operations: hands-on experience operating HPC/AI clusters and NVIDIA GPU-accelerated infrastructure—driver and CUDA toolkit lifecycle, firmware management, workload profiling, and troubleshooting across CPUs, GPUs and high‑speed interconnects.
• Kubernetes at Scale: extensive expertise in running Kubernetes in GPU-accelerated and HPC environments—node lifecycle, upgrades and draining, scheduler behaviour, batch schedulers such as Slurm, and mixed bare-metal/virtualised estates.
• Observability & Automation: Proficiency in Python and Bash, IaC and configuration management (Ansible, Terraform), GitOps-based fleet lifecycle and upgrade management, and observability stacks (Prometheus, Grafana, Loki) for monitoring, alerting and building fault-tolerant systems.
• Linux, Storage & Networking Fundamentals: Linux knowledge (RedHat, Ubuntu), OS tuning and security, data centre networking, Lustre/GPFS/ZFS/XFS.

Responsibilities

• Own Day 2 production stability and reliability at fleet scale across EMEA NCP partners: define, measure and improve MTBI and job goodput on multi-thousand-GPU clusters, and hold the error budget for the platforms you support.
• Build fault-detection to remediation loop—GPU, node, fabric and storage fault signatures, automated triage, node drain, cordon, reset and return-to-service workflows.
• Run preventive maintenance and proactive firmware and field-notice rollout campaigns across partner fleets, including risk assessment, staged rollout, maintenance-window planning and verification at scale.
• Own observability and reliability tooling for GPU fleets: Prometheus/Grafana/Loki, DCGM and XID diagnostics, node-level health agents such as GPUd and NVSentinel, and fleet-wide reliability intelligence—extending or integrating them into the partner’s own stack.
• Lead incident response and post-incident review with NVEX, TAM and partner teams: drive root-cause analysis, track corrective actions to closure, and convert recurring incidents into automation or health checks.
• Own Kubernetes node lifecycle and reliability engineering on heterogeneous open stacks—Kubernetes, KubeVirt, Slurm and GPU-aware schedulers—covering node health policy, upgrades, reboot orchestration, and safe workload draining.

Skills

Python

Bash

GitOps

Terraform

Ansible

Prometheus

Grafana

Loki

On-call

Education

BS/MS/PhD in CS/EE/Physics/Math or related

Equivalent experience

Tools

Kubernetes

Docker

KubeVirt

Slurm

DCGM

XID diagnostics

Job description

NVIDIA Corporation is seeking a Day 2 Site Reliability Engineer to join the EMEA NVIDIA Cloud Partner Operations team. You will own fleet-scale production reliability for multi-thousand-GPU deployments, defining MTBI, improving goodput and managing the error budget across partner estates.

The role emphasizes hands-on reliability, SLOs, and automation across open stack Kubernetes, KubeVirt, Slurm, and GPU tooling. Collaboration with NVEX, TAMs, and partner teams is key.