Senior Storage Reliability Engineer – Scale-Out AI Cloud

Lambda; San Francisco (CA); Hybrid; USD 267,000 - 356,000; Full time

Benefits offered by this job

Health, dental, and vision

Wellness stipend

401k with company match

Flexible PTO

Job summary

Lambda, The Superintelligence Cloud, is a leader in AI cloud infrastructure powering AI researchers to hyperscalers. Our Storage Engineering team builds the data platform and software-defined storage powering Lambda’s world-class services.

This role focuses on reliability, performance, capacity, monitoring, and automation. You’ll own incidents end to end, design self-healing systems, and collaborate with cross-functional teams to deploy storage at scale across data centers.

Qualifications

• 5+ years of experience operating Linux systems in production or HPC environments with hands-on storage experience at scale.
• Hands-on experience with Software-Defined Storage (SDS) platforms and integration with management and data-plane APIs.
• Strong incident-response instincts, owning storage incidents end to end from alert to postmortem.
• Experience with monitoring/logging platforms (Prometheus, Grafana, Alertmanager, Datadog, SumoLogic) and building dashboards/alerts.
• Kubernetes experience with GitOps tooling (ArgoCD, Helm/Kustomize) and troubleshooting.
• CI/CD tooling (GitHub Actions, Jenkins, BuildKite), containerization (Docker/Podman), and systems programming (Python or Go).
• Infrastructure as Code (Terraform, Ansible).
• Storage protocols across file, object, block, and structured storage.

Responsibilities

• Own the reliability, performance, and capacity health of Lambda's production storage fleet across all data centers.
• Build and maintain monitoring, dashboards, and alerting for storage performance, capacity, and hardware failures.
• Investigate and resolve storage-related incidents using telemetry, logs, and profiling.
• Automate ticketing, escalation, and incident-response workflows to reduce repetitive triage.
• Design self-healing automation for drive replacements, node swaps, and capacity rebalancing.
• Implement CI/CD pipelines for storage automation and tooling.
• Collaborate with Storage Engineers, Fleet Orchestration, and Release Engineering to deploy software-defined storage across sites.
• Work with hardware and networking teams to diagnose low-level I/O and network issues surface as storage symptoms.
• Participate in on-call rotation to drive down MTTR and reduce repeat paging.

Skills

Linux systems

Storage at scale

SDS platforms

Monitoring dashboards

Kubernetes

CI/CD tooling

IaC Terraform/Ansible

Networking I/O

Tools

Prometheus

Grafana

GitHub Actions

Jenkins

BuildKite

Docker/Podman

Ansible

Terraform

Job description

Lambda, The Superintelligence Cloud, is a leader in AI cloud infrastructure powering AI researchers to hyperscalers. Our Storage Engineering team builds the data platform and software-defined storage powering Lambda’s world-class services.

This role focuses on reliability, performance, capacity, monitoring, and automation. You’ll own incidents end to end, design self-healing systems, and collaborate with cross-functional teams to deploy storage at scale across data centers.