Staff Engineer: AI Storage & GPU Infrastructure

Together AI

United States

Remote

USD 250,000 - 330,000

Full time

10 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Startup equity
Health insurance
Remote work flexibility

Job summary

Together AI seeks a senior storage engineer to operate, scale, and optimize multi-petabyte storage systems for AI training and inference. You will build caching, data-movement layers, and GPU-direct delivery of model weights while integrating cutting-edge technologies.

You will design multi-tier caching, KV-cache offload, and enforce strong tenancy across thousands of nodes, with a focus on extreme throughput and low latency. Remote-friendly, highly performance-driven role.

Qualifications

  • 8+ years in storage engineering at multi-petabyte scale.

Responsibilities

  • Architect and implement the storage strategy and roadmap for Together AI at scale.
  • Engineer and scale multi-petabyte AI/ML storage systems by integrating Vast, Weka, Ceph.
  • Develop intelligent caching and tiered storage architectures for extreme IOPS and throughput at GPU scale.
  • Design KV-cache offload and reuse across GPU memory, host memory, NVMe, and distributed cache.
  • Tune storage isolation for secure multi-tenancy in production environments.
  • Code Kubernetes storage operators and controllers for automated provisioning and quota enforcement.
  • Optimize end-to-end data paths to achieve 10+ GB/s per GPU node and scale across thousands of nodes.
  • Contribute to open-source storage and inference projects with benchmarking and profiling.

Skills

Go
Python
C/C++ or Rust
Kubernetes
Terraform
Ansible
Helm
GitOps
RDMA/InfiniBand
Linux performance

Education

BS/MS in Computer Science or Engineering

Tools

Ceph
WekaFS
Lustre
Vast
GPFS
S3/MinIO
Kubernetes CSI drivers
Terraform/Ansible/Helm

Job description

Together AI seeks a senior storage engineer to operate, scale, and optimize multi-petabyte storage systems for AI training and inference. You will build caching, data-movement layers, and GPU-direct delivery of model weights while integrating cutting-edge technologies.

You will design multi-tier caching, KV-cache offload, and enforce strong tenancy across thousands of nodes, with a focus on extreme throughput and low latency. Remote-friendly, highly performance-driven role.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Engineer: GPU-Scale Distributed Storage & AI Infra
Staff Engineer: GPU-Scale Distributed Storage & AI Infra

Fdetimes • San Francisco (CA)

Remote
USD 250,000 - 330,000
Startup equity
Health insurance
Remote work flexibility
+1
Senior Storage Engineer — AI Infra, High-Perf Storage + Equity
Senior Storage Engineer — AI Infra, High-Perf Storage + Equity

Cotidal Limited • Palo Alto (CA)

On-site
USD 180,000 - 240,000
Medical, dental, and vision coverage
Unlimited PTO
Relocation support
Senior Storage Engineer - AI Infra & GPU Clusters (Remote)
Senior Storage Engineer - AI Infra & GPU Clusters (Remote)

Hamilton Barnes • San Francisco (CA)

Remote
USD 170,000 - 230,000
Stock options
Remote work allowance
Staff Engineer: Distributed Storage for AI & HPC
Staff Engineer: Distributed Storage for AI & HPC

Hidden Jobs • United States

Remote
USD 160,000 - 210,000
Storage Engineer (AI Infrastructure) - Hosting
Storage Engineer (AI Infrastructure) - Hosting

Hamilton Barnes • San Francisco (CA)

Remote
USD 170,000 - 230,000
Stock options
Remote work allowance
Senior AI Data Storage Engineer - Cloud-Native, Equity
Senior AI Data Storage Engineer - Cloud-Native, Equity

NVIDIA Corporation • North Carolina

On-site
USD 152,000 - 241,500
Equity compensation
Benefits
AI Storage Architect for High-Performance GPU Cloud
AI Storage Architect for High-Performance GPU Cloud

Bitdeer Technologies Group • Austin (TX)

On-site
USD 140,000 - 200,000
Senior Data Infrastructure Engineer (GPU/AI)
Senior Data Infrastructure Engineer (GPU/AI)

Fireworks AI, Inc. • San Mateo (CA)

On-site
USD 180,000 - 260,000
Senior Storage Systems Engineer, AI Data Infrastructure
Senior Storage Systems Engineer, AI Data Infrastructure

Digitailor • San Mateo (CA)

On-site
USD 210,000 - 270,000
Senior Storage Architect for AI Supercomputer (Remote)
Senior Storage Architect for AI Supercomputer (Remote)

Runware • Palo Alto (CA)

Remote
USD 190,000 - 240,000
Stock options
Remote-first culture
Flexible schedule
+4