Senior AI Infrastructure Engineer: Scalable GPU Clusters

KLA; Ann Arbor (MI); On-site; USD 130,000 - 190,000; Full time

Job summary

KLA is seeking a Sr. Software Engineer to design, build, and operate production-grade AI infrastructure for large-scale GPU training and inference in Ann Arbor.

You will own multi-node GPU clusters, orchestration, accelerator management, and performance engineering, with a strong emphasis on open-source integration and GitOps practices. The role requires deep Kubernetes expertise, production experience with Ray and AI frameworks, and robust Python/Linux skills.

Qualifications

• Bachelor's Degree and eight (8) years of Software Engineering experience.
• Four (4) years in software, cloud, platform, HPC, or systems engineering, including two (2) years supporting distributed AI/ML workloads.
• Proven hands-on experience building GPU clusters from the ground up and operating them at production scale.
• Deep Kubernetes expertise, including operators, CRDs, Helm, networking, storage, scheduling, and cluster lifecycle management.
• Production experience with Ray and at least two of the following: NVIDIA NeMo RL, vLLM, SGLang, or NVIDIA Dynamo.
• Strong knowledge of NVIDIA GPUs, CUDA, NCCL, GPU Operator, DCGM, MIG, RDMA, and multi-node collective communication.
• Experience debugging training and inference performance across GPU, CPU, memory, network, and storage layers.
• Proficiency in Python and Linux, plus experience with containers, CI/CD, GitOps, infrastructure-as-code, and platform observability.
• Demonstrated open-source contribution, maintainership, or meaningful participation in an AI infrastructure project.

Responsibilities

• Design and deploy scalable, multi-node GPU clusters on Kubernetes, including compute, networking, storage, scheduling, security, and observability.
• Build distributed training and reinforcement learning platforms using Ray and NVIDIA NeMo RL, supporting frameworks such as PyTorch and JAX.
• Deploy and optimize high-throughput LLM inference using vLLM, SGLang, and NVIDIA Dynamo.
• Implement GPU scheduling, quotas, isolation, autoscaling, health monitoring, and capacity management for multi-tenant environments.
• Profile and troubleshoot GPU workloads using NVIDIA Nsight Systems, Nsight Compute, DCGM, CUDA, NCCL, and related diagnostics.
• Automate cluster provisioning, upgrades, workload deployment, and operational recovery through infrastructure-as-code and GitOps practices.
• Contribute to or actively participate in relevant open-source AI infrastructure communities and bring upstream best practices into the platform.

Skills

Python

Linux

CI/CD

GitOps

Problem solving

Communication

Cloud

Distributed systems

Education

Bachelor's Degree in Software Engineering or related field

Tools

Kubernetes

Ray

NVIDIA NeMo RL

vLLM

SGLang

NVIDIA Dynamo

PyTorch

JAX

CUDA

NCCL

NVIDIA Nsight Systems

Nsight Compute

Job description

KLA is seeking a Sr. Software Engineer to design, build, and operate production-grade AI infrastructure for large-scale GPU training and inference in Ann Arbor.

You will own multi-node GPU clusters, orchestration, accelerator management, and performance engineering, with a strong emphasis on open-source integration and GitOps practices. The role requires deep Kubernetes expertise, production experience with Ray and AI frameworks, and robust Python/Linux skills.