GPU DevOps / Infrastructure Setup Engineer

Cinergy Technology Inc; Hyderabad; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Cinergy Technology Inc is seeking an experienced GPU DevOps / Infrastructure Setup Engineer to design, deploy, and maintain GPU-enabled AI and HPC infrastructure in a hybrid on‑prem/cloud environment. You will drive cluster bring‑up, driver setup, and GPU workload deployment.

You will work with BCM, CUDA, and container orchestration (Docker/Kubernetes) while optimizing performance and reliability. Strong scripting and automation skills are essential.

Qualifications

• 3+ years in DevOps, SRE, or infrastructure engineering.
• Experience with GPU environments in real-world deployments.
• Ability to explain infrastructure, GPU hardware used, deployments, and issues resolved.

Responsibilities

• Set up, configure, and maintain GPU-enabled servers and AI infrastructure.
• Deploy on-premises GPU compute clusters and verify provisioning.
• Lead end-to-end GPU cluster bring-up: provisioning, configuration, validation, testing.
• Configure NVIDIA BCM for cluster deployment and management where applicable.
• Install and configure OS, GPU drivers, CUDA Toolkit, libraries, and software.
• Deploy GPU workloads on bare-metal, cloud, or hybrid environments.
• Configure GPU-enabled Docker containers and Kubernetes workloads.
• Deploy and manage LLM applications using frameworks like vLLM, Ollama, HF TGI, or NVIDIA Triton.
• Troubleshoot GPU hardware, driver compatibility, CUDA initialization, and workload issues.
• Support InfiniBand networking and high-speed interconnects for multi-GPU/multi-node setups.
• Automate provisioning, configuration management, and deployment with scripting and tools.
• Develop and maintain CI/CD pipelines for infra and app deployments.
• Monitor GPU utilization, memory, temp, power, and performance; implement alerting.
• Optimize GPU utilization and inference performance; perform capacity planning and upgrades.
• Maintain technical docs, install guides, and troubleshooting records.
• Collaborate with AI/ML, development, platform, and infra teams to support production workloads.

Skills

GPU server setup

Linux administration

CUDA

Docker

Kubernetes

CI/CD

Prometheus & Grafana

Tools

NVIDIA GPU drivers

NVIDIA BCM

CUDA Toolkit

InfiniBand

Terraform

Ansible

Job description

We are looking for an experienced GPU DevOps / Infrastructure Setup Engineer to build, configure, deploy, and maintain GPU-enabled infrastructure for AI/ML and High-Performance Computing (HPC) workloads.

The ideal candidate will have hands-on experience with GPU server setup, on-premises AI infrastructure, Linux administration, NVIDIA GPU drivers, CUDA, containerization, infrastructure automation, and GPU workload deployment.

Experience with NVIDIA Base Command Manager (BCM), GPU cluster bring-up, InfiniBand networking, and LLM application deployment will be highly preferred. Candidates with experience across different GPU hardware vendors and cloud, on-premises, or hybrid environments are welcome.

Key Responsibilities

• Set up, configure, and maintain GPU-enabled servers, compute nodes, and AI infrastructure.
• Build and deploy on-premises AI infrastructure and GPU compute clusters.
• Perform end-to-end GPU cluster bring-up, including provisioning, configuration, validation, and testing.
• Configure and manage NVIDIA Base Command Manager (BCM) for cluster deployment and management, where applicable.
• Install and configure operating systems, GPU drivers, CUDA Toolkit, libraries, and supporting software.
• Deploy and maintain GPU workloads on bare-metal servers, cloud platforms, and hybrid environments.
• Configure GPU-enabled Docker containers and Kubernetes workloads.
• Deploy and manage LLM applications using frameworks such as vLLM, Ollama, Hugging Face Text Generation Inference (TGI), or NVIDIA Triton.
• Configure and troubleshoot GPU hardware, driver compatibility, CUDA initialization, memory utilization, and workload execution issues.
• Support InfiniBand networking and high-speed interconnects for multi-GPU or multi-node environments.
• Automate infrastructure provisioning, configuration management, and deployment using appropriate scripting and automation tools.
• Develop and maintain CI/CD pipelines for infrastructure and application deployments.
• Monitor GPU utilization, memory, temperature, power consumption, system health, and performance.
• Implement monitoring, logging, alerting, health checks, and automated recovery using tools such as Prometheus and Grafana.
• Optimize GPU utilization, VRAM consumption, inference latency, throughput, and overall infrastructure performance.
• Troubleshoot Linux, networking, storage, container, hardware, and application deployment issues.
• Perform infrastructure upgrades, patching, maintenance, capacity planning, and resource scaling.
• Maintain technical documentation, installation procedures, configuration guides, and troubleshooting records.
• Collaborate with AI/ML, development, platform, and infrastructure teams to support production workloads.

Required Technical Skills

GPU Infrastructure and Hardware

• Hands-on experience with GPU server setup, configuration, and operations.
• Knowledge of GPU hardware, server components, PCIe, power requirements, cooling, and hardware compatibility.
• Experience with GPU models such as NVIDIA A100, H100, H200, L40S, RTX series, or equivalent GPU platforms.

Linux, Drivers, and CUDA

• Strong Linux administration and troubleshooting skills.
• Experience installing and configuring GPU drivers, CUDA Toolkit, and supporting libraries.
• Ability to validate GPU detection, driver functionality, CUDA compatibility, and workload execution.

AI Infrastructure and HPC

• Experience building or maintaining on-premises AI infrastructure.
• Understanding of GPU cluster provisioning, compute nodes, and cluster validation.
• Experience with NVIDIA Base Command Manager (BCM) is preferred.
• Experience deploying on-premises GPU infrastructure without BCM is also valuable.
• Knowledge of InfiniBand, RDMA, and high-speed networking for GPU clusters is an advantage.

DevOps and Containerization

• Experience with Docker and GPU-enabled containers.
• Kubernetes experience for GPU workload deployment and orchestration.
• Infrastructure automation using Bash, Python, Ansible, Terraform, or equivalent tools.
• Familiarity with CI/CD pipelines, Git, configuration management, and infrastructure monitoring.

LLM Deployment and Performance

• Experience deploying LLM applications on GPU-enabled cloud instances or on-premises GPU servers.
• Familiarity with vLLM, Ollama, Hugging Face TGI, NVIDIA Triton, or equivalent model-serving frameworks.
• Understanding of API-based model serving and GPU resource allocation.
• Experience troubleshooting GPU memory issues and optimizing inference performance.
• Familiarity with Prometheus, Grafana, NVIDIA SMI, or DCGM for GPU monitoring.

Experience Requirements

• 3+ years of experience in DevOps, SRE, platform engineering, or infrastructure engineering.
• At least 1-2 years of hands-on experience working with GPU environments.
• Demonstrated experience in at least one real-world GPU infrastructure or AI deployment project.
• Ability to explain the infrastructure architecture, GPU hardware used, individual responsibilities, deployment process, and technical issues resolved.
• Experience supporting production environments is preferred.

Preferred Qualifications

• Hands-on InfiniBand configuration and troubleshooting.
• Bare-metal GPU server installation and configuration without Docker.
• multi-GPU or multi-node cluster deployment.
• Production LLM serving and GPU performance optimization.
• Experience across on-premises, cloud, and hybrid infrastructure environments.

Candidate Profile

We are looking for engineers with practical infrastructure experience who can independently configure GPU environments, troubleshoot complex technical issues, and support reliable AI/ML and HPC deployments.

Candidates must be able to explain real projects they have worked on, the GPU hardware and tools used, their individual contributions, and the challenges they successfully resolved.