AI Infrastructure System Engineer

Jobgether

India

On-site

INR 1,500,000 - 2,800,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Large-scale GPU infrastructure
Challenging distributed systems
Automation and software development
Cross-disciplinary collaboration
Professional growth

Job summary

Jobgether partner company seeks an AI Infrastructure System Engineer in India to design and operate large-scale GPU infrastructure for AI model training and inference.

You will build software-driven systems to automate the lifecycle of GPU clusters, from provisioning to retirement, and develop intelligent platforms that improve availability, performance, and reliability across thousands of accelerators.

Qualifications

  • 3+ years building distributed systems or large-scale backend software.
  • Strong software engineering in Python, Go or Rust.
  • Experience developing automation platforms or developer infrastructure.
  • Familiarity with Linux and infra tech like Kubernetes, Terraform and Ansible.
  • Strong systems-thinking abilities across hardware and software.
  • Automation-first mindset focused on eliminating repetitive work.

Responsibilities

  • Design and develop fleet automation systems to provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention.
  • Build AI infrastructure agents to automate deployments, identify root causes of failures, support incident triage, and autonomous remediation.
  • Develop fleet intelligence platforms to monitor hardware health, firmware, networking, storage, thermal conditions, and workload performance for predictive failure detection.
  • Create software solutions that maximize GPU availability, utilization, performance, and reliability across thousands of accelerators.
  • Build automated validation systems for GPUs, InfiniBand and RoCE networks, NVLink and NVSwitch technologies, storage infrastructure, and distributed AI workloads.
  • Develop internal platforms and tools enabling infrastructure teams to manage systems programmatically.
  • Continuously improve deployment speed, operational efficiency, and infrastructure reliability via automation and system optimization.
  • Collaborate with hardware, networking, platform, and AI teams to advance large-scale AI infrastructure capabilities.

Skills

Distributed systems
Backend software
Python/Go/Rust
Automation platforms
Linux
Kubernetes
Terraform
Ansible
Systems thinking
Automation mindset

Tools

Kubernetes
Terraform
Ansible

Job description

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for an AI Infrastructure System Engineer based in India.

Join a team building and operating large-scale GPU infrastructure that powers advanced AI model training and inference. In this role, you will design software-driven systems to automate the entire lifecycle of GPU clusters, from provisioning and deployment to monitoring, repair, and retirement. You will develop intelligent automation and fleet management platforms that improve infrastructure availability, performance, and reliability across thousands of accelerators. Working at the intersection of software, hardware, networking, and AI, you will tackle complex distributed systems challenges at an exceptional scale. This is an opportunity for an engineer who prefers building systems over manual operations and is passionate about eliminating repetitive work through automation. Your contributions will help make large-scale AI infrastructure more autonomous, efficient, and resilient.

Accountabilities
  • Design and develop fleet automation systems to provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention.

  • Build AI infrastructure agents that automate deployments, identify root causes of failures, support incident triage, and execute autonomous remediation.

  • Develop fleet intelligence platforms to monitor hardware health, firmware, networking, storage, thermal conditions, and workload performance, enabling predictive failure detection.

  • Create software solutions that maximize GPU availability, utilization, performance, and reliability across large-scale accelerator fleets.

  • Build automated validation systems for GPUs, InfiniBand and RoCE networks, NVLink and NVSwitch technologies, storage infrastructure, and distributed AI workloads.

  • Develop internal platforms and tools that enable infrastructure teams to manage systems programmatically rather than relying on manual operations.

  • Continuously improve deployment speed, operational efficiency, and infrastructure reliability through automation and system optimization.

  • Collaborate with hardware, networking, platform, and AI engineering teams to advance the capabilities of large-scale AI infrastructure.

Requirements:
  • At least 3 years of experience building distributed systems, infrastructure platforms, or large-scale backend software.

  • Strong software engineering skills in at least one of the following languages: Python, Go, or Rust.

  • Practical experience developing platforms, automation systems, or developer infrastructure.

  • Familiarity with Linux and infrastructure technologies such as Kubernetes, Terraform, Ansible, or similar tools.

  • Strong systems-thinking abilities, with the capacity to understand and troubleshoot challenges spanning both hardware and software.

  • A passion for solving complex infrastructure problems through software engineering.

  • An automation-first mindset, with a strong preference for building systems that eliminate repetitive tasks and manual processes.

Bonus Experience:
  • Experience with GPU infrastructure, CUDA, NCCL, NVLink, or NVSwitch.

  • Knowledge of InfiniBand or RoCE networking technologies.

  • Experience with bare-metal provisioning and infrastructure lifecycle management.

  • Exposure to large-scale AI training or inference clusters.

  • Familiarity with hardware health monitoring and predictive failure detection.

  • Experience with distributed storage systems.

  • Knowledge of AI agents and autonomous infrastructure operations.

Benefits:
  • Opportunity to work on large-scale GPU infrastructure supporting advanced AI training and inference.

  • Exposure to complex engineering challenges across distributed systems, hardware, networking, and AI platforms.

  • Opportunity to build software and automation systems operating across thousands of accelerators.

  • Close collaboration with multidisciplinary engineering teams.

  • Professional growth through hands-on experience with emerging AI infrastructure technologies.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Infrastructure System Engineer
AI Infrastructure System Engineer

AItoolnavio • India

Remote
INR 2,600,000 - 4,500,000
Large-scale GPU infra exposure
Cross-disciplinary collaboration
Hands-on AI platforms
+2
AI Infrastructure System Engineer Bangalore
AI Infrastructure System Engineer Bangalore

The AI Landscape • Bengaluru

On-site
INR 800,000 - 1,200,000
Senior Software Engineer — Infra Agent Systems Remote India
Senior Software Engineer — Infra Agent Systems Remote India

FDE Times • India

Remote
INR 3,500,000 - 7,000,000
Senior Engineer Specialist – GPU Cluster & AI Infrastructure
Senior Engineer Specialist – GPU Cluster & AI Infrastructure

Vivantify Technology Solutions India Pvt. Ltd. • Chennai

On-site
INR 2,000,000 - 3,000,000
Competitive salary
Professional growth
Collaborative environment
+1
Senior Software Engineer — Infra Agent Systems Remote India India
Senior Software Engineer — Infra Agent Systems Remote India India

Together Computer Inc • India

Remote
INR 400,000 - 700,000
Specialist in machine learning engineering
Specialist in machine learning engineering

Jobgether • India

Remote
INR 1,500,000 - 2,500,000
Fully remote work
Full-time employment
Work on large-scale AI infrastructure
+1
Senior AI Infrastructure Engineer - DGX Cloud, Senior AI Infrastructure Engineer - DGX Cloud
Senior AI Infrastructure Engineer - DGX Cloud, Senior AI Infrastructure Engineer - DGX Cloud

NVIDIA Corporation • Bengaluru

On-site
INR 1,500,000 - 2,500,000
Senior AI Infrastructure Engineer - DGX Cloud
Senior AI Infrastructure Engineer - DGX Cloud

IN01 NVIDIA Graphics Bengaluru • Bengaluru

On-site
INR 1,800,000 - 2,400,000
Senior Software Engineer, AI Infrastructure Solutions
Senior Software Engineer, AI Infrastructure Solutions

IN02 NVIDIA GraphicsPLtd,Pune • Pune District

On-site
INR 4,000,000 - 6,000,000
Senior Engineer Specialist – GPU Cluster & AI Infrastructure
Senior Engineer Specialist – GPU Cluster & AI Infrastructure

Vivantify Technology Solutions India Private Limited • Chennai

On-site
INR 1,400,000 - 2,800,000
Competitive salary
Growth opportunities
Collaborative environment
+1