AI Infra Engineer: GPU Clusters & HPC

Veeda AI; California; On-site; USD 150,000 - 210,000; Full time

Job summary

Veeda AI in the United States (California) is assembling a small, high‑impact team to architect and operate scalable HPC infrastructure for cutting‑edge multimodal AI workloads. You’ll own GPU clusters, networking, storage, and security, ensuring reliable data delivery and observability across systems.

Ideal candidates have deep Linux and HPC experience, plus automation skills and familiarity with PyTorch/NCCL, Slurm, and cloud PoCs.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, or equivalent hands-on experience in HPC or infrastructure engineering.
• Strong troubleshooting skills in low-level Linux networking, kernel and PCIe/NUMA tuning, hardware diagnostics, and distributed storage (NFS, NVMe-oF, Lustre, Ceph).
• Proficient in automation and infrastructure-as-code tools (e.g., Ansible, Terraform, Helm, Python/Bash scripting), and treats cluster configuration as reviewed, version-controlled code.
• Experience supporting distributed deep learning workloads (PyTorch/NCCL, Ray, DeepSpeed) closely enough to distinguish infrastructure faults from model bugs with controlled benchmarks.
• Experience administering Linux-based HPC clusters (Slurm/Kubernetes) or cloud PoCs and security policy management.

Responsibilities

• GPU Cluster Operations: Design, deploy, and operate Slurm-on-Kubernetes GPU clusters, including compute health, interconnects, storage, and scheduling.
• Access Control and Security: Design and operate identity provisioning and access control for source code, cluster, data storages, and dev tools.
• Network and Data: Design and maintain distributed data transmission and caching services for just-in-time data delivery to clusters.
• Observability & Hardware Health: Build telemetry pipelines (Prometheus, Grafana, DCGM, Redfish) to detect hardware issues and coordinate with cloud vendors.
• Resource Forecasting: Work with Model and Data teams to determine resource needs and lead PoC rounds with vendors.
• Infra-OPEX: Prepare operational expenses forecasts and provide expert opinion on infra spending decisions.
• End-to-End Ownership: Lead projects through the full software lifecycle including CI/CD and production observability.

Skills

HPC administration

Linux networking

Automation tooling

Distributed DL workloads

Security policies

Education

Bachelor's in CS/CE or equivalent

Tools

Slurm

Kubernetes

GitLab CI

Terraform

Ansible

Job description

Veeda AI in the United States (California) is assembling a small, high‑impact team to architect and operate scalable HPC infrastructure for cutting‑edge multimodal AI workloads. You’ll own GPU clusters, networking, storage, and security, ensuring reliable data delivery and observability across systems.

Ideal candidates have deep Linux and HPC experience, plus automation skills and familiarity with PyTorch/NCCL, Slurm, and cloud PoCs.