Senior Storage Engineer

Runware; United States; Remote; USD 180.000 - 260.000; Vollzeit

Benefits dieser Stelle

Stock options

Remote-first culture

Flexible schedule

Generous PTO

Parental leave

Company retreats

Zusammenfassung

Runware is building AI infrastructure to scale its 1 MW Sonic Inference Pod fleet across the US and Europe. This Senior Storage Engineer role owns the storage architecture for model weights, caches and customer data, from object storage to node NVMe memory, and sets the standard for storage across all pods.

You will design data paths, optimize loads, and work with inference, platform and hardware teams to deliver low-latency, highly durable storage at scale.

Qualifikationen

• 8+ years in storage or systems engineering with hands-on scale
• Built storage systems from design to production at petabyte scale
• Deep Linux storage expertise: block layer, io_uring, NVMe, XFS/ext4/ZFS
• Experience with Lustre/GPFS/Ceph or similar and S3-compatible object storage at petabyte scale
• Strong knowledge of NVMe, NVMe-oF, RDMA networking
• Fluent in Go, Rust or C/C++ and Python; capable of building tooling
• Experience using AI to automate benchmarking, scripts and tooling
• Willingness to travel to US and Europe as needed

Aufgaben

• Own end-to-end storage architecture for the fleet, including object storage and per-pod NVMe cache
• Design and build the model distribution path and warming strategies for new pods
• Profile data paths from disk to GPU memory and remove bottlenecks
• Select and operate storage systems and validate choices on hardware
• Set reliability and durability targets: capacity planning, monitoring, recovery
• Create automated playbooks for pod bring-up and storage provisioning
• Collaborate with engine, platform and hardware teams to scale storage
• Use AI to accelerate benchmarking, tooling and triage
• Ensure data residency, encryption at rest and in-country placement

Kenntnisse

Storage engineering

Linux

Go

Rust

C/C++

Python

System design

AI tooling

Travel readiness

Tools

NVMe

NVMe-oF

RDMA

Lustre

Ceph

GPFS

WEKA

VAST

S3-compatible storage

Jobbeschreibung

Runware is the AI inference platform that builds and runs its own infrastructure, end to end. Our Sonic Inference Engine has served more than 20 billion requests for over 1 million developers and 500 million end users, with 400,000+ models behind a single integration, at up to 10x lower cost per token than traditional providers.

We are now building the AI Factory that runs it. The Sonic Inference Pod packs 1 MW of custom-designed compute into a 20-foot container with closed-loop, water-free cooling, sited wherever power exists. Pods are live in the US and Europe, and we are scaling to dozens of pods and tens of thousands of GPUs, all operated as one distributed fabric that reroutes traffic the moment a pod goes offline announcement.

Every one of those 400,000 models has to be on the right GPU, in the right pod, in seconds. Storage is what makes that possible. That is the job.

The role

As Senior Storage Engineer, you will own the storage architecture of Runware's AI supercomputer: how model weights, caches and customer data move from object storage on two continents to the NVMe and GPU memory of the node that needs them. You will design the data path, build the first version with your own hands, and set the standard for how storage is run across every pod.

This is a hands‑on senior individual‑contributor role. You will specify the systems, benchmark them on real hardware, find the reason a 70 GB checkpoint took four minutes to load instead of twenty seconds, and then fix it at the root so it never happens again. You will work directly with our inference engine, platform and hardware teams, with our storage and NVMe vendors, and with the site teams deploying pods.

Success means a cold model loads onto a GPU anywhere in the fleet in seconds, a new pod has its working set warm before it takes traffic, and a disk, node or whole site failing is invisible to the developer calling our API.

What you'll do

• Own the end-to-end storage architecture for the fleet: the durable object store that holds 400,000+ models, the per-pod high-throughput tier that feeds GPUs, the node-local NVMe cache, and the replication and placement logic that ties them together across the US and Europe.
• Design and build the model distribution path. Decide how weights get from the source of truth to a pod, how they are chunked, deduplicated, prefetched and pinned, and how a new pod warms its working set before it takes production traffic.
• Drive model load latency down and keep it down. Profile the full path from disk to GPU memory, including the kernel, file system, NVMe-oF or RDMA transport, GPUDirect Storage and the inference engine's loader, and remove the bottleneck wherever it sits.
• Select, benchmark and operate the storage systems themselves: parallel and distributed file systems, object storage, NVMe drives and enclosures, and the software that manages them. Validate every choice on our hardware before it ships to a pod.
• Set and hit reliability and durability targets. Own capacity planning, monitoring, alerting, drive health and failure prediction, rebuild behaviour, backup and recovery, and on-call for the storage layer.
• Make storage a routine part of pod bring‑up: a documented, automated playbook so that the storage tier of the next 1 MW pod is provisioned, validated and warm in hours, not weeks.
• Partner with inference engine, platform and hardware engineering so that what we design is operable at scale, and feed what you learn from production back into the next generation of servers and pods.
• Use AI to accelerate the work: automate benchmarks, runbooks, triage and tooling with LLM-based agents, so that a very small team can run storage for a very large fleet.
• Own data residency and the operational side of storage security: encryption at rest, access control, retention and in-country placement for sovereign deployments.

Requirements

What we're looking for

• 8+ years in storage or systems engineering, with several of those designing and running storage at a company that operates its own hardware at scale.
• A track record of building storage systems, not just operating what someone else built. You have taken a design from a whiteboard to petabytes in production and know what goes wrong between the two.
• Deep, current, hands‑on Linux storage expertise: the block layer, io_uring and NVMe queues, file systems such as XFS, ext4 and ZFS, the page cache, and the tools to see exactly where an I/O spends its time.
• Real experience with at least one parallel or distributed file system in production (Lustre, WEKA, VAST, GPFS, Ceph or similar) and with S3‑compatible object storage at petabyte scale.
• Strong understanding of NVMe, NVMe-oF, RDMA transports and the networking that storage rides on. You can tell a saturated PCIe link from a slow drive from a misconfigured switch.
• Fluent in a systems language (Go, Rust or C/C++) and Python, and comfortable writing the tooling, benchmarks and automation yourself rather than waiting for someone else to.
• You use AI to accelerate your own output every day, for scripting, benchmarking, analysis, documentation and debugging, and you have a clear view of how a small team armed with these tools should operate differently.
• Comfortable owning outcomes in a fast‑moving, ambiguous environment, making decisions with incomplete information and being accountable for latency, durability and cost.
• Able to travel to sites in the US and Europe as needed.

Nice to have

• Experience feeding large GPU fleets for AI training or inference: GPUDirect Storage, NCCL‑aware data loading, model weight caching and the loaders in vLLM, SGLang, TensorRT-LLM or similar engines.
• Hands‑on experience with InfiniBand or RoCE fabrics and 400G+ Ethernet from the storage side.
• Experience with content‑addressed or deduplicated storage, erasure coding, and multi‑region replication and consistency trade‑offs.
• Kernel or driver contributions, or deep familiarity with SPDK, DPDK or user‑space storage stacks.
• Experience with Kubernetes CSI, Ceph or Rook on bare metal, and an automation stack such as Ansible and Terraform with Prometheus and Grafana observability.
• Familiarity with the storage hardware supply chain: drive qualification, firmware lifecycle, enclosure and backplane design, spares and RMA economics.
• Experience building or running storage to meet data residency, sovereignty or compliance requirements across jurisdictions.

Benefits

What we offer

• A rare mandate: design the storage layer of an AI supercomputer from the ground up, at a company whose business depends on doing it better than anyone else, with the hardware budget and backing to match.
• Meaningful stock options.
• We're a remote‑first collective, meeting in person twice a year to plan, brainstorm and celebrate wins. We keep core hours for collaboration and calls; outside those, you set your own schedule.
• Our release cycles are fast and intense, and they're followed by real downtime. We expect you to take it.
• Generous paid time off, including vacation, sick days and public holidays.
• Paid maternity, paternity and caregiver leave.
• Twice‑yearly company retreats in inspiring locations.