Senior Storage Engineer -AI Infrastructure

ARC Cloud; San Jose (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

ARC Cloud in San Jose, CA builds and operates high-throughput storage systems to feed AI at scale, from parallel filesystems to object storage for datasets. Your work will support thousands of GPUs and demanding checkpoint traffic.

You design, deploy, and operate Lustre, Ceph, WEKA, or GPFS clusters; tune I/O for checkpointing and data loading; manage tiering across NVMe, SSD and HDD; and ensure DR with backups, snapshots, and replication.

Qualifications

• 5+ years storage engineering in large-scale environments.
• Production experience with parallel filesystems (Lustre, Ceph, WEKA, GPFS).
• Understanding AI workload I/O patterns.
• Experience with object storage (S3-compatible) at scale.
• Scripting in Python or Bash for storage automation.

Responsibilities

• Design, deploy, and operate parallel/distributed filesystems for AI training workloads.
• Tune storage performance for checkpointing, data loading, and multi-reader training patterns.
• Manage storage capacity planning, tiering, and lifecycle across NVMe, SSD, and HDD tiers.
• Own backup, snapshot, replication, and disaster recovery for cluster storage.
• Monitor storage health, throughput, and latency; drive proactive maintenance.
• Integrate storage with Slurm/Kubernetes workflows and data pipelines.
• Evaluate new storage technologies and vendors for next-generation clusters.
• Maintain runbooks and documentation for storage operations.

Skills

Storage engineering

Parallel filesystems

Scripting (Python/Bash)

AI workload I/O patterns

Object storage

Tools

Lustre

Ceph

WEKA

GPFS

Job description

Build and operate the high-throughput storage systems feeds AI at scale — from parallel filesystems serving checkpoint traffic to object storage for datasets. Your storage must keep pace with thousands of GPUs.

Key Responsibilities

• Design, deploy, and operate parallel/distributed filesystems (WEKA, Lustre, Ceph, or GPFS) for AI training workloads
• Tune storage performance for checkpointing, data loading, and multi-reader training patterns
• Manage storage capacity planning, tiering, and lifecycle across NVMe, SSD, and HDD tiers
• Own backup, snapshot, replication, and disaster recovery for cluster storage
• Monitor storage health, throughput, and latency; drive proactive maintenance
• Integrate storage with Slurm/Kubernetes workflows and data pipelines
• Evaluate new storage technologies and vendors for next-generation clusters
• Maintain runbooks and documentation for storage operations

Required Qualifications

• 5+ years of storage engineering in large-scale environments
• Production experience with at least one parallel filesystem: Lustre, Ceph, WEKA, or GPFS
• Understanding of AI workload I/O patterns (checkpointing, data loaders)
• Experience with object storage (S3-compatible) at scale
• Scripting in Python/Bash for storage automation

Preferred Qualifications

• RDMA/NVMe-oF storage networking experience
• Experience with storage for 1,000+ GPU clusters
• Familiarity with VAST Data, DDN, or Pure Storage platforms