Senior Staff GPU Infra Engineer - AI Supercomputer Scale

Runware

United States

Remote

USD 250,000 - 400,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Stock options
Remote-first culture
Flexible hours
Paid time off

Job summary

Runware is seeking a Senior Staff Engineer, Infrastructure Systems, to own the GPU software stack across all Runware nodes. This IC role combines firmware, kernel, driver, CUDA runtime, libraries, and container tooling with deep collaboration across operations and inference teams.

You will drive driver/CUDA qualification, diagnostics, and automated health of GPU fleets, while shaping the next generation of AI compute at scale.

Qualifications

  • 12+ years building and operating systems software, with several on GPU software in production at scale.
  • Deep, hands-on knowledge of the NVIDIA software stack: kernel driver, CUDA runtime and libraries, NVML, DCGM, Container Toolkit.
  • Proven ability to root-cause GPU failures across hardware, firmware, driver, runtime and application layers.
  • Strong Linux internals: kernel, drivers, PCIe, NUMA, IOMMU, containers and perf tools like ftrace and eBPF.

Responsibilities

  • Own the GPU software baseline: kernel, NVIDIA driver, firmware, CUDA runtime and libraries, and container toolkit.
  • Qualify every driver and CUDA release; maintain a compatibility matrix across GPU generations and model families.
  • Root-cause GPU failures: diagnose Xid errors, memory and fragmentation issues, and misbehaving CUDA calls.
  • Build GPU acceptance and diagnostics, including burn-in, DCGM tests, and real-workload verification.
  • Make GPUs visible and self-healing by centralizing telemetry and automating drain, reset, and replacement paths.
  • Control GPU exposure to workloads via container runtimes, device plugins, and MIG/sharing policies on Kubernetes and bare metal.
  • Bring up new GPU generations and verify NIC/NVLink validation for accelerators across racks.
  • Characterize hardware with real workloads and feed results into placement, pricing, and purchasing decisions.
  • Collaborate with NVIDIA and open-source projects; report bugs with reproducible cases and track changes.
  • Leverage AI to accelerate outputs, including automated triage, test generation, and runbook automation.

Skills

GPU software
Linux kernel
Python
C/C++
CUDA
Kubernetes
DCGM
NVIDIA stack

Tools

Docker
Container Toolkit
NCCL
NVML

Job description

Career Trajectory: "Senior Staff Engineer" is a Principal/Distinguished-equivalent level at most companies — this is a high-leverage individual contributor role, not a management track. It signals Runware values deep technical ownership. For engineers who want to stay hands-on while having outsized impact, this is a strong fit.

Compensation Context: No salary is listed, but Senior Staff Engineers with GPU/CUDA specialization in the US typically command $250,000–$400,000+ total compensation at AI infrastructure companies (base + equity). The stock options mentioned could be highly valuable given Runware's growth stage in the AI compute market.

High-Demand Skill Development: This role sits at the intersection of the most sought-after skills in tech right now:

  • GPU infrastructure at scale (NVIDIA CUDA, DCGM, MIG, NVLink)
  • AI inference optimization (TensorRT, vLLM, LLM serving)
  • eBPF, Linux kernel internals — rare and extremely valuable

Green Flags: Remote-first culture with async flexibility, fast release cadence, direct collaboration with NVIDIA, and a mandate to use AI tooling to multiply output — all signal a forward-thinking engineering culture. The 12+ year experience bar keeps the team senior and competent.

Red Flags to Watch: "Fast-moving, ambiguous environment" and 24/7 service ownership at fleet scale can mean high on-call pressure. Clarify incident rotation expectations and SLA obligations before accepting. The company is early-stage enough that processes may still be maturing.

As Senior Staff Engineer, Infrastructure Systems, you will own the GPU software stack on every Runware node: the firmware, the kernel and NVIDIA driver, the CUDA runtime and libraries, the container runtime, and the way Kubernetes and our inference services see and share a GPU. When a GPU misbehaves anywhere in the fleet, you will find out why and write the software that keeps it from happening again.

This is a senior staff-level individual contributor role for someone who knows GPUs from the driver up and still writes code every day. One month that means qualifying a new driver and CUDA release against every model family we serve. The next it is the GPU part of the acceptance suite that every node must pass, a detector that catches a failing GPU from its Xid and ECC history before a customer does, the bring-up of a new GPU generation, or a CUDA error that turns out to start in firmware, a PCIe link or a kernel update. You will work directly with the node operations, Managed Kubernetes, inference performance and Sonic Inference Engine teams, and be the person each of them calls when the problem sits below the framework.

Success means that every GPU in every pod runs a known, tested combination of firmware, driver and CUDA, that upgrades roll across the fleet without failed inferences, that GPU faults are detected, classified and remediated automatically, and that a new GPU generation goes from delivery to serving traffic in days.

What You'll Do
  • Own the GPU software baseline. Versioned, reproducible combinations of kernel, NVIDIA driver, firmware, CUDA runtime and libraries (cuDNN, cuBLAS, NCCL, TensorRT) and container toolkit, tested before they roll and rolled across the fleet without downtime.
  • Qualify every driver and CUDA release. Keep the compatibility matrix across GPU generations, frameworks and model families, and catch regressions in correctness, performance and memory use before they reach production.
  • Take GPU failures to root cause: Xid errors, CUDA "unknown error", cuBLAS handle failures, out-of-memory and fragmentation, GPUs that fall off the bus. Separate hardware, firmware, driver, runtime and application causes, and prove which one it is.
  • Build GPU acceptance and diagnostics: the burn-in, DCGM diagnostics, memory and interconnect tests and real-workload tests that gate every node and rack before it serves traffic, and the reports customers receive at handover.
  • Make GPU health visible and self-healing. Pull telemetry from DCGM, the driver, the kernel log and the BMC into one trusted view, classify faults, and automate the drain, reset and replacement paths.
  • Own how GPUs are exposed to workloads: the container runtime, device plugins and the GPU Operator, MIG and sharing policies, and isolation between tenants, on bare metal and on Kubernetes.
  • Bring up new GPU generations and new pods with the hardware and node operations teams: first driver and firmware, NVLink and NVSwitch validation, and the automation that makes the next bring-up take days.
  • Characterize hardware with real workloads. Benchmark image, video, audio and LLM inference on each GPU generation and software stack, and feed the results into placement, pricing and the next purchase.
  • Work with NVIDIA and upstream projects. File driver and firmware bugs with reproducible cases and see them through, track what each release changes, and contribute fixes to the open-source tools we depend on.
  • Use AI to accelerate the work: agents for log and Xid triage, test generation, release qualification and runbook automation, so that the team's output grows faster than its headcount.
What We're Looking For
  • 12+ years building and operating systems software, with several of those on GPU software running in production at scale.
  • Deep, hands-on knowledge of the NVIDIA software stack: the kernel driver and its modules, the CUDA runtime and libraries, NVML, DCGM and the Container Toolkit, and how their versions interact.
  • A record of taking GPU failures to root cause across hardware, firmware, driver, runtime and application: Xid errors, ECC and row remapping, PCIe and NVLink errors, thermal and memory faults.
  • Strong Linux internals: the kernel, device drivers and modules, PCIe, NUMA and IOMMU, cgroups and containers, and the tools (perf, eBPF, ftrace) that show what a machine is really doing.
  • You write code every day. Fluent in Python and in at least one systems language (C, C++, Rust or Go), and comfortable enough in CUDA to write a test or reproduce a bug yourself.
  • Experience running GPUs under Kubernetes and on bare metal: device plugins, the GPU Operator, MIG, image builds and provisioning at scale.
  • Working knowledge of how inference frameworks use a GPU (PyTorch, TensorRT, ONNX Runtime): memory allocation, compilation and caches, and multi-GPU execution.
  • Clear technical writing. Your design documents, postmortems and ticket updates can be acted on by someone who was not in the room.
  • You use AI to accelerate your own output every day, and you have a clear view of how a small team armed with these tools should operate differently.
  • Comfortable owning outcomes in a fast-moving, ambiguous environment, picking the problem that matters most and being accountable for the result.
Nice to Have
  • Time spent building GPU drivers, CUDA libraries or GPU system software at a GPU vendor, a cloud provider or an HPC centre.
  • Experience with Blackwell-generation hardware (B200, GB200/GB300 NVL72), NVLink and NVSwitch topologies, and liquid-cooled racks.
  • Multi-GPU and multi-node execution: NCCL, GPUDirect, and RDMA over RoCE or InfiniBand.
  • Firmware qualification across BIOS, BMC, NIC and GPU, and acceptance testing with OEMs and ODMs.
  • Inference serving internals: PyTorch, TensorRT-LLM, vLLM, SGLang or custom diffusion and video pipelines, and how they load, schedule and fail.
  • Experience with non-NVIDIA accelerators (AMD or custom silicon) and their software stacks.
  • An open-source project you wrote or maintain that people run in production.
  • Multi-tenant isolation and confidential computing on shared GPU hardware.
  • Experience running a 24x7 service with published service levels.
Why Runware

You will design the network for an AI supercomputer that is being built right now, with the budget, the hardware, and the mandate to do it properly. Few network architects get to own a fabric from the first pod to the global backbone, with a direct line to the people deciding what the next pod looks like.

We're a remote-first collective, meeting in person twice a year to plan, brainstorm, celebrate wins, and enjoy some face-to-face time. We have core hours for cooperative working and calls, but outside of that your calendar is yours. Work the hours that let you perform at your peak while also building a healthy life.

Our release cycles are fast and intense, but they're followed by real downtime. After big pushes we expect the team to unplug, recharge, and come back ready and stronger than ever for the next leap.

  • Generous paid time off: vacation, sick days, public holidays
  • Meaningful stock options: share in the upside you create
  • Flexible hours: own your schedule outside core collaboration blocks
  • Family leave: paid maternity, paternity, and caregiver time
  • Company retreats: twice-yearly gatherings in inspiring locations

Is this role fully remote or does it require being in a specific location? Runware is a remote-first company. The role does have core collaboration hours, but outside of those your schedule is your own. The company meets in person twice a year for retreats. What level of GPU experience is required for this position? The role requires 12+ years of systems software experience, with several specifically on GPU software running in production at scale, plus deep hands-on knowledge of the full NVIDIA software stack including kernel drivers, CUDA, DCGM, and the Container Toolkit. Does the role involve people management or is it purely individual contributor? This is explicitly described as a senior staff-level individual contributor (IC) role — there is no mention of direct reports or management responsibilities. What coding languages are expected for day-to-day work? Fluency in Python is required, along with at least one systems language such as C, C++, Rust, or Go. You should also be comfortable enough in CUDA to write tests or reproduce bugs independently. Are stock options part of the compensation package? Yes, Runware offers meaningful stock options as part of its benefits, described as a way to "share in the upside you create." No specific salary figures are disclosed in the posting.

Remote work & career advice that actually works.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Staff Engineer - Infrastructure Systems
Senior Staff Engineer - Infrastructure Systems

Speedrun Talent Network • California

Remote
USD 180,000 - 240,000
Generous PTO
Stock options
Flexible hours
+2
Senior Staff Engineer - Infrastructure Systems
Senior Staff Engineer - Infrastructure Systems

Runware • California

Remote
USD 180,000 - 300,000
Generous paid time off
Meaningful stock options
Flexible hours
+2
Senior Staff Engineer - Infrastructure Systems
Senior Staff Engineer - Infrastructure Systems

present of AI • California

On-site
USD 180,000 - 240,000
R Open role · via a16z Speedrun 1d ago Senior Staff Engineer - Infrastructure Systems Runware (Picfinder) US INFRA Express interest →
R Open role · via a16z Speedrun 1d ago Senior Staff Engineer - Infrastructure Systems Runware (Picfinder) US INFRA Express interest →

JoinStartup • California

Remote
USD 180,000 - 260,000
AI Factory Network Architect
AI Factory Network Architect

Speedrun Talent Network • United States

Remote
USD 260,000 - 360,000
Generous paid time off
Stock options
Flexible hours
+2
AI Factory Network Architect
AI Factory Network Architect

Runware • Palo Alto (CA)

Remote
USD 280,000 - 360,000
Generous paid time off
Stock options
Flexible hours
+2
Senior Staff Engineer - Infrastructure Systems
Senior Staff Engineer - Infrastructure Systems

Runware • United States

On-site
USD 250,000 - 320,000
Network Architect
Network Architect

Speedrun Talent Network • United States

Remote
USD 250,000 - 360,000
PTO
Stock options
Flexible hours
+2
Network Architect
Network Architect

Runware • Palo Alto (CA)

Hybrid
USD 210,000 - 320,000
Generous paid time off
Meaningful stock options
Flexible hours
+2
Network Architect
Network Architect

Runware • San Francisco (CA)

Remote
USD 180,000 - 300,000
Generous paid time off
Meaningful stock options
Flexible hours
+2