Frontier AI Storage Architect

Prime Intellect; United States; On-site; USD 150,000 - 300,000; Full time

Job summary

Prime Intellect is building open frontier AI infrastructure, focusing on storage systems that feed frontier AI workloads. You will own reliable, high-throughput access to datasets, checkpoints, and model artifacts, ensuring performance, durability, availability, and cost balance as GPU clusters scale.

Role involves designing and operating storage architectures, deploying parallel filesystems, benchmarking, and automating capacity management.

Qualifications

• 3+ years building or operating production distributed storage systems.
• Hands-on experience with at least one parallel or distributed filesystem or object storage platform (Lustre, BeeGFS, Ceph, GPFS).
• Strong Linux administration and performance troubleshooting skills.
• Experience automating infrastructure operations in Python, Go, Bash, or similar languages.
• Understanding of storage failure modes, data integrity, replication, and recovery.

Responsibilities

• Design and operate storage architectures for training datasets, checkpoints, and research workflows.
• Deploy and tune parallel filesystems, object storage, and local NVMe caching for AI workloads.
• Benchmark throughput, latency, and metadata performance for training and checkpoint workloads.
• Build provisioning, capacity planning, lifecycle management, and automation for storage services.
• Design and test replication, recovery, backup, and failure-handling procedures with durability/availability targets.
• Diagnose performance and reliability issues across applications, clients, networks, filesystems, and devices.
• Implement access controls, tenant separation, quotas, monitoring, and runbooks; collaborate with compute and networking teams.

Skills

Distributed storage

Linux administration

Python

Go

Bash

Performance troubleshooting

Tools

Lustre

BeeGFS

Ceph

GPFS

Job description

Prime Intellect is building open frontier AI infrastructure, focusing on storage systems that feed frontier AI workloads. You will own reliable, high-throughput access to datasets, checkpoints, and model artifacts, ensuring performance, durability, availability, and cost balance as GPU clusters scale.

Role involves designing and operating storage architectures, deploying parallel filesystems, benchmarking, and automating capacity management.