Network Engineer

Larsen & Toubro-Vyoma; Chennai District; On-site; INR 1,500,000 - 3,000,000; Full time

Job summary

Larsen & Toubro-Vyoma seeks a seasoned DC networking engineer to design, deploy and operate ultra-low-latency Fabrics for GPU training at scale. You will tune InfiniBand HDR/NDR, RoCE networks, and implement QoS for training, inference, and control planes.

Responsibilities include leading P0/P1 incident response, automating configurations with Ansible and Terraform, and driving reliability improvements across the GPU infrastructure stack.

Qualifications

• 7–15 years DC networking experience
• Deep InfiniBand (HDR/NDR) and RoCE tuning at scale
• GPU cluster communications patterns

Responsibilities

• Lead deployment and operation of ultra-low-latency network fabrics for GPU training
• Lead P0/P1 incident response for NVIDIA GPU infrastructure
• Automate configuration with Ansible and Terraform
• Validate NCCL performance and optimize all-reduce patterns
• Implement QoS for training vs inference vs control-plane
• Design and implement runbooks for path failures and degradation
• Ensure secure multi-tenancy with BGP/EVPN and ACLs

Skills

InfiniBand HDR/NDR

RoCE tuning

NCCL performance

GPU cluster communications

QoS implementation

Education

BE/BTech in CS or ECE

Tools

Ansible

Terraform

Job description

Deliver and operate ultra-low-latency, high-bisection network fabrics for GPU training at ~10K GPU scale with deterministic performance.

Implementation

Build IB (HDR/NDR) east-west fabrics and Ethernet north-south connectivity; leaf-spine with ECMP.

Deploy and tune RoCE v2, PFC/ECN, MTU, buffer profiles; IB routing/credit/timers; SHARP/UCX offloads where applicable.

Operations

Day-2 fabric health (congestion hotspots, slow-drain, link flaps); automate config with Ansible/Terraform.

Lifecycle of switch OS/firmware; link-level optics and cable qualification; inventory and spares.

Performance & Optimization

Validate NCCL collectives performance across pods; troubleshoot all-reduce bottlenecks and imbalance.

Implement QoS classes for training vs inference vs control-plane.

Reliability & Incident

Lead P0/P1 critical incident response for NVIDIA GPU infrastructure, including InfiniBand, RoCE, Ethernet fabric and AI/ML cluster environments.

Troubleshot performance bottlenecks involving InfiniBand fabrics, RoCE networks, congestion management, NCCL communications, GPUDirect RDMA, and distributed AI training workloads.

Led post-incident reviews and implemented automation, monitoring enhancements, and architectural improvements to increase overall cluster availability, stability, and performance.

Create runbooks for path failures, fabric partitions, and credit starvation; design graceful degradation.

Security & Compliance

BGP/EVPN multi-tenancy, ACLs, role-based access to fabric managers; config signing/change control.

Experience & Educational Requirement

BE/B-Tech or equivalent with Computer Science or Electronics & Communication

Certifications:- NVIDIA Networking; CCNP/CCIE DC; JNCIP-DC / Arista ACE.

RELEVANT EXPERIENCE

• 7-15 years DC networking; deep InfiniBand (HDR/NDR) and RoCE tuning at scale; GPU cluster comms patterns.

Tools / Tech