Staff Engineer, Distributed GPU Clusters

Kindredventures; San Francisco (CA); On-site; USD 140,000 - 230,000; Full time

Job summary

Kindredventures is building a Large Physics foundation Model and seeks an infrastructure engineer to design, deploy, and operate its GPU-driven compute environment. You will enable research at scale by provisioning, upgrading, and optimizing distributed clusters that power training and inference workloads.

You will extend orchestration, implement topology-aware scheduling, and deliver a self-serve platform for researchers, with strong focus on reliability, observability, and cost efficiency.

Qualifications

• Experience operating large-scale GPU clusters and orchestration tools.
• Strong systems background: Linux, networking, storage.
• Knowledge of cloud platforms (GCP, AWS, or Azure).
• Understanding of monitoring, logging, observability, and version control for ML systems.
• Familiarity with CUDA/NCCL for distributed workloads.
• Owns deliverables end-to-end, from requirements through autonomous execution.

Responsibilities

• Design, deploy, and operate large distributed GPU clusters end to end: provisioning, imaging, upgrades, and capacity planning.
• Extend scheduling and orchestration systems for topology-aware placement, preemption, quotas, and multi-tenancy across training and inference workloads.
• Build software that abstracts cluster management and presents a unified, self-serve interface to researchers and engineers.
• Own cluster storage and artifact paths for checkpoints and logs, with clear retention and lineage.
• Monitor and continuously improve reliability and error recovery; build observability to catch failures before researchers do.
• Partner with researchers to unblock large-scale runs and advise on performance and placement trade-offs.

Skills

GPU clusters

Container orchestration

Systems background

Cloud platforms

Monitoring observability

CUDA/NCCL

End-to-end ownership

Tools

Kubernetes

Slurm

Docker

Job description

Kindredventures is building a Large Physics foundation Model and seeks an infrastructure engineer to design, deploy, and operate its GPU-driven compute environment. You will enable research at scale by provisioning, upgrading, and optimizing distributed clusters that power training and inference workloads.

You will extend orchestration, implement topology-aware scheduling, and deliver a self-serve platform for researchers, with strong focus on reliability, observability, and cost efficiency.