A complete application in a minute
Tailored resume and cover letter, ready to send.
L&T Technology Services Ltd. in Mumbai is seeking an experienced GPU Infrastructure Engineer to build and operate large-scale GPU compute pods for high-throughput AI training and inference across a multi-rack 10K GPU cluster.
Expect deep work with GPGPU, CUDA, MIG/vGPU, Slurm and Kubernetes device scheduling to deliver predictable, low-latency performance. You will lead capacity planning, performance tuning, and incident management, coordinate with Networking, GPU Ops, Platform Eng, and Storage
LNT/GIE-HE/1920234
End Date 09 May 2027
Required Experience 6 - 10 years
GPGPU CUDA Artificial Intelligence HPC
Bachelor of Technology (BTech)
Build and operate large?scale GPU compute pods to deliver predictable, high?throughput, low?latency training and inference services across 10K GPU cluster.
Stand up multi?pod GPU clusters (rack/power/cooling layouts; TOR/leaf connectivity; IB/Ethernet host configs).
Implement GPU partitioning (MIG/vGPU profiles) and quota policies for multi?tenant environments.
Integrate cluster schedulers (Slurm/Kubernetes) with GPU device plugins, node feature discovery, and accounting/quotas.
Own day?2 operations across firmware/driver/DCGM/NVML lifecycles; execute change windows with zero/low downtime.
Capacity planning (GPU/CPU/Memory/NIC) and bin?packing strategies for heterogeneous GPU SKUs.
Tune NCCL/UCX, GPU clocks/persistence, GPU Direct Storage, NUMA/locality, and CUDA runtime parameters.
Drive benchmarking and acceptance (HPL, HPL?AI, MLPerf?like internal suites); track perf regressions with SLOs.
Lead P0/P1 incident response for GPU, CUDA, or scheduler issues; perform root?cause and preventative actions.
Act as the primary technical lead during production outages, coordinating cross-functional teams across Networking, GPU Operations, Platform Engineering, Storage, and Application teams to restore services within SLA targets.
Perform detailed Root Cause Analysis (RCA) for network, GPU, CUDA, NCCL, RDMA, and scheduler-related failures, identifying underlying causes and implementing preventive and corrective actions.
Collaborate with platform and GPU engineering teams to resolve issues impacting CUDA jobs, Kubernetes scheduling, Slurm workload management, GPU resource allocation, and large-scale AI training environments.
Define golden images; implement node remediation (cordon/drain/reimage) and auto?healing workflows.
Enforce GPU tenancy isolation (MIG, cgroup/device cgroup, mpsd), secure drivers/containers, SBOM and image scanning.
Publish runbooks, performance baselines, and application tuning guides for LLM training and inference.
Experience & Educational Requirement
BE/B-Tech or equivalent with Computer Science or Electronics & Communication
RELEVANT EXPERIENCE
Tools / Tech