Infrastructure / Cluster Engineer

Linuxcareers; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

Linuxcareers is seeking an Infrastructure/Cluster Engineer to design and operate large-scale clusters that enable AI inference at scale. The role focuses on managing diverse hardware architectures and building robust infrastructure.

The ideal candidate will possess deep expertise in Linux systems, automation tools, and orchestration technologies. Responsibilities include debugging performance issues and designing observability systems for cluster health. Experience with GPU infrastructure is a plus.

Qualifications

• Experience in infrastructure, cluster engineering, platform engineering, or distributed systems.
• Deep Linux systems experience with an emphasis on debugging.
• Strong automation skills using Terraform, Ansible, or Python.

Responsibilities

• Design and operate large-scale CPU and GPU clusters.
• Build automation solutions for infrastructure management.
• Debug complex production issues across various technology layers.

Skills

Infrastructure engineering

Linux systems

Kubernetes

Automation with Terraform

GPU infrastructure

Tools

Ansible

Python

Go

Job description

Gimlet is building AI infrastructure and orchestration platforms for large-scale AI datacenters. This Infrastructure/Cluster Engineer role involves designing, building, and operating heterogeneous cluster infrastructure that intelligently routes workloads across diverse hardware architectures to enable production AI inference at scale.

What You'll Do

• Design, deploy, and operate large-scale CPU, GPU, and accelerator clusters powering production AI inference workloads
• Build automation for provisioning, configuration, upgrades, validation, and lifecycle management across heterogeneous bare-metal infrastructure
• Debug complex production issues spanning Linux, networking, storage, drivers, firmware, and orchestration layers
• Build and operate high-performance networking infrastructure including RDMA-enabled environments and accelerator interconnects
• Design and scale observability systems for cluster health, capacity, performance, failures, and workload behavior

What You Need

• Experience in infrastructure, cluster engineering, platform engineering, SRE, HPC, or distributed systems
• Deep Linux systems experience including debugging performance, networking, storage, processes, and kernel-level issues
• Experience operating Kubernetes, Slurm, Nomad, or similar orchestration and scheduling systems
• Strong automation skills using Terraform, Ansible, Helm, Python, Go, or equivalent
• Experience with GPU or accelerator infrastructure including drivers, firmware, CUDA/ROCm stacks, or hardware validation

Nice to Have

• Experience building or operating AI inference, training, HPC, or neocloud infrastructure
• Experience with bare-metal provisioning, PXE/iPXE, image pipelines, BIOS/firmware management, or rack bring-up
• Experience with multi-tenant cluster isolation, quota systems, fair scheduling, or usage accounting
• Experience building observability platforms using Prometheus, OpenTelemetry, Grafana, or similar technologies