Storage Infrastructure Engineer for AI/ML at Scale

Lightningai; Greater London; Hybrid; GBP 133,000 - 163,000; Full time

Benefits offered by this job

Health coverage

Equity

Retirement matching (US/UK)

Flexible PTO

Winter break

Parental leave

Learning allowance

Wellness stipend

Sabbatical

Hybrid work

Office meals

Job summary

Lightning AI is seeking a Storage Infrastructure Engineer to build and operate storage systems for large-scale AI training and inference. You will work at the intersection of software, hardware, and operations, developing automation and scaling distributed storage across bare-metal infrastructure.

The role involves owning the data plane of storage, ensuring high throughput and low latency, with VAST and Ceph experience valued.

Qualifications

• 5+ years of experience in infrastructure/systems engineering.
• Hands-on experience operating distributed storage systems (e.g., VAST, Ceph).
• Strong Linux systems experience in production environments.
• Proficiency in Python or similar scripting/programming languages for automation.
• Experience with bare-metal infrastructure and hardware-oriented systems.
• Ability to debug complex issues across storage, OS, hardware, and networking.
• Experience with storage networking protocols (e.g., NFS).
• Experience with capacity planning, monitoring, and performance tuning.

Responsibilities

• Operate and scale distributed storage systems (VAST, Ceph).
• Improve performance, reliability, and efficiency of storage systems for AI workloads.
• Troubleshoot complex storage and data path issues across layers.
• Optimize storage performance for high-throughput, low-latency workloads.
• Build and maintain automation for provisioning, managing, and monitoring storage infrastructure.
• Develop Python-based tools to reduce manual operational overhead.
• Manage lifecycle of storage clusters from deployment to scaling.
• Operate Linux-based production systems, including bare-metal environments.

Skills

Python scripting

Linux systems

Storage systems

Performance tuning

Troubleshooting

Networking basics

Automation

Tools

VAST

Ceph

NFS

S3-compatible storage

RDMA

Job description

Lightning AI is seeking a Storage Infrastructure Engineer to build and operate storage systems for large-scale AI training and inference. You will work at the intersection of software, hardware, and operations, developing automation and scaling distributed storage across bare-metal infrastructure.

The role involves owning the data plane of storage, ensuring high throughput and low latency, with VAST and Ceph experience valued.