Senior Storage Engineer

Runware

Palo Alto (CA)

Remote

USD 190,000 - 240,000

Full time

11 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Stock options
Remote-first culture
Flexible schedule
Downtime after releases
Paid time off
Parental leave
Company retreats

Job summary

Runware is building a world‑class AI inference platform with an integrated storage fabric. We seek a Senior Storage Engineer to own the storage architecture for our AI supercomputing fleet, including model weights, caches and data movement across continents.

This is a hands-on IC role, partnering with inference, platform and hardware teams to ship performant storage at scale. You will drive latency down, validate hardware stacks, and implement automated benchmarks and tooling using AI to

Qualifications

  • 8+ years in storage or systems engineering, with hands-on experience at scale.
  • Experience designing and running storage systems end-to-end.
  • Experience with Linux storage internals and NVMe technologies.
  • Production experience with parallel or distributed file systems and object storage.

Responsibilities

  • Own the end-to-end storage architecture for the fleet: durable object store, per-pod tier, and node-local cache across US and Europe.
  • Design and build the model distribution path, including chunking, deduplication, and warm-up of working set.
  • Profile and optimize the full disk-to-GPU path to minimize latency and remove bottlenecks.
  • Evaluate and operate storage systems: parallel/file systems, object stores, NVMe enclosures, and related software.
  • Set reliability and durability targets: capacity planning, monitoring, backup, and on-call for storage.

Skills

Linux storage
NVMe/storage hardware
Go / Rust / C/C++
Python
Distributed storage
S3-compatible storage
AI tooling / scripting
Travel willingness

Job description

Runware is the AI inference platform that builds and runs its own infrastructure, end to end. Our Sonic Inference Engine has served more than 20 billion requests for over 1 million developers and 500 million end users, with 400,000+ models behind a single integration, at up to 10x lower cost per token than traditional providers.

We are now building the AI Factory that runs it. The Sonic Inference Pod packs 1 MW of custom-designed compute into a 20-foot container with closed-loop, water-free cooling, sited wherever power exists. Pods are live in the US and Europe, and we are scaling to dozens of pods and tens of thousands of GPUs, all operated as one distributed fabric that reroutes traffic the moment a pod goes offline (announcement).

Every one of those 400,000 models has to be on the right GPU, in the right pod, in seconds. Storage is what makes that possible. That is the job.

The role

As Senior Storage Engineer, you will own the storage architecture of Runware’s AI supercomputer: how model weights, caches and customer data move from object storage on two continents to the NVMe and GPU memory of the node that needs them. You will design the data path, build the first version with your own hands, and set the standard for how storage is run across every pod.

This is a hands-on senior individual-contributor role. You will specify the systems, benchmark them on real hardware, find the reason a 70 GB checkpoint took four minutes to load instead of twenty seconds, and then fix it at the root so it never happens again. You will work directly with our inference engine, platform and hardware teams, with our storage and NVMe vendors, and with the site teams deploying pods.

Success means a cold model loads onto a GPU anywhere in the fleet in seconds, a new pod has its working set warm before it takes traffic, and a disk, node or whole site failing is invisible to the developer calling our API.

What you’ll do
  • Own the end-to-end storage architecture for the fleet: the durable object store that holds 400,000+ models, the per-pod high-throughput tier that feeds GPUs, the node-local NVMe cache, and the replication and placement logic that ties them together across the US and Europe.
  • Design and build the model distribution path. Decide how weights get from the source of truth to a pod, how they are chunked, deduplicated, prefetched and pinned, and how a new pod warms its working set before it takes production traffic.
  • Drive model load latency down and keep it down. Profile the full path from disk to GPU memory, including the kernel, file system, NVMe‑oF or RDMA transport, GPUDirect Storage and the inference engine’s loader, and remove the bottleneck wherever it sits.
  • Select, benchmark and operate the storage systems themselves: parallel and distributed file systems, object storage, NVMe drives and enclosures, and the software that manages them. Validate every choice on our hardware before it ships to a pod.
  • Set and hit reliability and durability targets. Own capacity planning, monitoring, alerting, drive health and failure prediction, rebuild behaviour, backup and recovery, and on‑call for the storage layer.
  • Make storage a routine part of pod bring‑up: a documented, automated playbook so that the storage tier of the next 1 MW pod is provisioned, validated and warm in hours, not weeks.
  • Partner with inference engine, platform and hardware engineering so that what we design is operable at scale, and feed what you learn from production back into the next generation of servers and pods.
  • Use AI to accelerate the work: automate benchmarks, runbooks, triage and tooling with LLM‑based agents, so that a very small team can run storage for a very large fleet.
  • Own data residency and the operational side of storage security: encryption at rest, access control, retention and in‑country placement for sovereign deployments.
What we’re looking for
  • 8+ years in storage or systems engineering, with several of those designing and running storage at a company that operates its own hardware at scale.
  • A track record of building storage systems, not just operating what someone else built. You have taken a design from a whiteboard to petabytes in production and know what goes wrong between the two.
  • Deep, current, hands‑on Linux storage expertise: the block layer, io_uring and NVMe queues, file systems such as XFS, ext4 and ZFS, the page cache, and the tools to see exactly where an I/O spends its time.
  • Real experience with at least one parallel or distributed file system in production (Lustre, WEKA, VAST, GPFS, Ceph or similar) and with S3‑compatible object storage at petabyte scale.
  • Strong understanding of NVMe, NVMe‑oF, RDMA transports and the networking that storage rides on. You can tell a saturated PCIe link from a slow drive from a misconfigured switch.
  • Fluent in a systems language (Go, Rust or C/C++) and Python, and comfortable writing the tooling, benchmarks and automation yourself rather than waiting for someone else to.
  • You use AI to accelerate your own output every day, for scripting, benchmarking, analysis, documentation and debugging, and you have a clear view of how a small team armed with these tools should operate differently.
  • Comfortable owning outcomes in a fast‑moving, ambiguous environment, making decisions with incomplete information and being accountable for latency, durability and cost.
  • Able to travel to sites in the US and Europe as needed.
Nice to have
  • Experience feeding large GPU fleets for AI training or inference: GPUDirect Storage, NCCL‑aware data loading, model weight caching and the loaders in vLLM, SGLang, TensorRT‑LLM or similar engines.
  • Hands‑on experience with InfiniBand or RoCE fabrics and 400G+ Ethernet from the storage side.
  • Experience with content‑addressed or deduplicated storage, erasure coding, and multi‑region replication and consistency trade‑offs.
  • Kernel or driver contributions, or deep familiarity with SPDK, DPDK or user‑space storage stacks.
  • Experience with Kubernetes CSI, Ceph or Rook on bare metal, and an automation stack such as Ansible and Terraform with Prometheus and Grafana observability.
  • Familiarity with the storage hardware supply chain: drive qualification, firmware lifecycle, enclosure and backplane design, spares and RMA economics.
  • Experience building or running storage to meet data residency, sovereignty or compliance requirements across jurisdictions.
What we offer
  • A rare mandate: design the storage layer of an AI supercomputer from the ground up, at a company whose business depends on doing it better than anyone else, with the hardware budget and backing to match.
  • Meaningful stock options.
  • We’re a remote‑first collective, meeting in person twice a year to plan, brainstorm and celebrate wins. We keep core hours for collaboration and calls; outside those, you set your own schedule.
  • Our release cycles are fast and intense, and they’re followed by real downtime. We expect you to take it.
  • Generous paid time off, including vacation, sick days and public holidays.
  • Paid maternity, paternity and caregiver leave.
  • Twice‑yearly company retreats in inspiring locations.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Storage Engineer
Senior Storage Engineer

Runware • United States

Remote
USD 180,000 - 240,000
Head of Infrastructure Operations
Head of Infrastructure Operations

Speedrun Talent Network • United States

Remote
USD 180,000 - 280,000
Meaningful stock options
Remote-first culture
Generous PTO and holidays
+2
Head of Infrastructure Operations
Head of Infrastructure Operations

Runware • San Jose (CA)

Remote
USD 250,000 - 360,000
Stock options
Remote-first culture
Generous PTO and holidays
+1
Head of Infrastructure Operations
Head of Infrastructure Operations

Runware • San Francisco (CA)

Remote
USD 210,000 - 320,000
Stock options
Remote-first culture
Generous PTO and paid family leave
+1
Senior Security Engineer
Senior Security Engineer

Runware • Palo Alto (CA)

Remote
USD 180,000 - 260,000
Stock options
Remote-first work model
Generous paid time off
+2
Head of Infrastructure Operations
Head of Infrastructure Operations

Runware • Palo Alto (CA)

Remote
USD 240,000 - 310,000
Stock options
Remote-friendly culture
Twice-yearly company retreats
Principal Engineer, Model Performance
Principal Engineer, Model Performance

Speedrun Talent Network • United States

Remote
USD 250,000 - 350,000
Stock options
Remote-first culture
Generous PTO
+2
Senior Staff Engineer - Infrastructure Systems
Senior Staff Engineer - Infrastructure Systems

Speedrun Talent Network • California

Remote
USD 180,000 - 240,000
Generous PTO
Stock options
Flexible hours
+2
Principal Engineer, Model Performance
Principal Engineer, Model Performance

Runware • United States

Remote
USD 250,000 - 420,000
Stock options
Remote-first
Generous PTO
+2
Principal Engineer, Model Performance
Principal Engineer, Model Performance

Runware • San Francisco (CA)

Remote
USD 180,000 - 280,000
Stock options
Remote-first culture
Flexible schedule
+1