Senior Storage Reliability Engineer - AI Cloud Infra

Lambda Inc.; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Wellness stipend

Commuter stipend

401k match

Job summary

Lambda Inc. in San Francisco is seeking a Storage Engineer to own the reliability, performance, and capacity of our production storage fleet across multiple data centers, using a software-defined data plane.

You will build monitoring, dashboards, and alerting for storage performance and hardware failures, and automate workflows to speed root-cause analysis. A strong background in SDS, Kubernetes, CI/CD, and IaC is essential, with on-call responsibilities and deep telemetry-driven incident

Qualifications

• 5+ years of Linux production/HPC experience with scale-out storage on software-defined platforms.
• Hands-on experience with Software-Defined Storage (SDS) platforms at scale.
• Strong incident-response instincts from alert to postmortem.
• Experience with monitoring/logging platforms (Prometheus, Grafana, Alertmanager, Datadog, SumoLogic).
• Kubernetes experience with GitOps tooling (ArgoCD, Helm/Kustomize).
• CI/CD tooling (GitHub Actions, Jenkins, BuildKite), containerization (Docker/Podman), and Python or Go.
• Infrastructure as Code (Terraform, Ansible).
• Storage protocols across file (NFS/SMB), object (S3), block (NVMe-oF/TCP), and structured storage (vector DB, SQL).

Responsibilities

• Own reliability, performance, and capacity health of Lambda's production storage fleet across data centers.
• Build and maintain monitoring, dashboards, and alerting for storage performance, capacity, and hardware failures.
• Investigate and resolve storage incidents using telemetry, logs, and performance profiling, from NIC issues to cluster rebuilds.
• Automate ticketing, escalation, and incident-response workflows to reduce redundant triage.
• Design and maintain self-healing automation for drive replacement, node swaps, rebuild monitoring, and capacity rebalancing.
• Implement CI/CD pipelines for storage automation and tooling.
• Collaborate with Storage Engineers, Fleet Orchestration, and Release Engineering to deploy and configure software-defined storage across sites using Ansible, Jenkins, etc.
• Work with hardware and networking teams to diagnose low-level I/O and network issues surface as storage symptoms.
• Participate in on-call rotation to reduce MTTR and build automation to avoid repeat paging.

Skills

Linux admin

Scale-out storage

SDS platforms

Monitoring dashboards

Kubernetes

CI/CD tooling

IaC tools

Networking troubleshooting

Python/Go

Jenkins/GitHub Actions

Tools

Prometheus

Grafana

Alertmanager

Datadog

SumoLogic

ArgoCD

Helm/Kustomize

Docker/Podman

Terraform

Ansible

Job description

Lambda Inc. in San Francisco is seeking a Storage Engineer to own the reliability, performance, and capacity of our production storage fleet across multiple data centers, using a software-defined data plane.

You will build monitoring, dashboards, and alerting for storage performance and hardware failures, and automate workflows to speed root-cause analysis. A strong background in SDS, Kubernetes, CI/CD, and IaC is essential, with on-call responsibilities and deep telemetry-driven incident