Senior Software Engineer, AI Infrastructure Solutions

NVIDIA Corporation

India

On-site

INR 1,500,000 - 2,100,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

NVIDIA Corporation is seeking a Senior Software Engineer to join the AI Infrastructure Solutions team in Pune. You will design and operate AI infrastructure for LLM-based applications, develop multi-agent workflows, and integrate LLMs via APIs and inference endpoints.

You will containerize with Docker, deploy on Kubernetes, and ensure production readiness with strong observability. Requirements include 5+ years in software engineering with 2+ years in AI/LLM solutions, strong Python, and

Qualifications

  • Bachelor's degree in CS, Engineering or related field.
  • 5+ years of software engineering experience, with 2+ years in AI/LLM-based solutions.
  • Hands-on experience building agentic AI and autonomous agent systems.

Responsibilities

  • Design, build, and operate AI infrastructure solutions for LLM-based applications.
  • Develop agentic AI workflows and multi-agent systems.
  • Integrate and serve LLMs via APIs and inference endpoints; optimize latency and cost.
  • Containerize apps with Docker; deploy on Kubernetes with Helm charts and GPU workloads.
  • Own production readiness, troubleshooting, performance tuning, and escalation for L3 issues.
  • Define observability for AI systems: logging, metrics, tracing, monitoring.

Skills

Python
C#/.NET Core
Docker
Kubernetes
AI/LLM
REST APIs
CI/CD
Observability

Education

Bachelor's degree in CS/Engineering

Tools

GitHub Actions
Azure DevOps
GitLab CI
Jenkins

Job description

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It's a unique legacy of innovation that's fueled by great technology and amazing people. NVIDIA is seeking an outstanding Senior Software Engineer to join our AI Infrastructure Solutions team in Pune. This is your opportunity to work on groundbreaking technologies and help build the future of computing! NVIDIA is a global leader in innovation, offering a dynamic and inclusive work environment in Pune or equivalent experience. We value diversity and provide equal opportunities to all candidates.

What You'll Be Doing:
  • Design, build, and operate AI infrastructure solutions that power LLM-based applications and autonomous agents across the enterprise.
  • Develop agentic AI workflows and multi-agent systems, including tool calling, planning, memory, retrieval-augmented generation (RAG), and orchestration using frameworks such as LangChain, LangGraph, or similar.
  • Integrate and serve LLMs through APIs and inference endpoints, optimizing for latency, throughput, cost, and reliability.
  • Containerize applications with Docker and deploy and scale them on Kubernetes, including Helm charts, autoscaling, and GPU-enabled workloads.
  • Take applications to L3 level: own production readiness, advanced troubleshooting, root-cause analysis, performance tuning, and escalation support for complex issues beyond L1/L2.
  • Define and maintain observability for AI systems, including logging, metrics, tracing, and monitoring of agent behavior, LLM quality, and system health.
  • Build and maintain CI/CD pipelines for automated build, test, and deployment of AI applications and infrastructure.
  • Develop high-performance backend services and RESTful APIs (Python and/or .NET Core) that expose AI capabilities to internal applications.
What We Need to See:
  • Bachelor's or equivalent experience in Computer Science, Engineering, or a related field.
  • 5+ years of software engineering experience, with 2+ years working on AI/LLM-based solutions.
  • Hands-on experience building agentic AI and autonomous agent systems, including tool use, multi-agent orchestration, and RAG pipelines.
  • Strong experience with LLMs and their integration (OpenAI-compatible APIs, NVIDIA NIM, Hugging Face, or similar), prompt engineering, and evaluation.
  • Strong proficiency in Python; experience with C#/.NET Core and ASP.NET Core for REST API development is a plus.
  • Solid experience with Docker and Kubernetes, including deploying, scaling, and troubleshooting containerized workloads in production.
  • Proven experience supporting applications at L3 level, including incident management, debugging, performance optimization, and root-cause analysis.
  • Experience with CI/CD tools (GitHub Actions, Azure DevOps, GitLab CI, Jenkins) and Git-based workflows.
Ways to Stand Out from the Crowd:
  • Experience with GPU-accelerated workloads, NVIDIA AI Enterprise, NVIDIA NIM, Triton Inference Server, or NeMo.
  • Familiarity with MCP (Model Context Protocol) and agent-to-agent communication patterns.
  • Experience with LLM evaluation, guardrails, and safety frameworks.
  • Contributions to open-source AI or infrastructure projects.

NVIDIA delivers competitive pay, a wide range of benefits, and the possibility to contribute to groundbreaking technologies. Widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Software Engineer, AI Infrastructure Solutions
Senior Software Engineer, AI Infrastructure Solutions

NVIDIA Corporation • Pune District

On-site
INR 2,500,000 - 4,500,000
Senior Software Engineer, AI Infrastructure Solutions
Senior Software Engineer, AI Infrastructure Solutions

IN02 NVIDIA GraphicsPLtd,Pune • Pune District

On-site
INR 4,000,000 - 6,000,000
Senior System Software Engineer - Local AI Automation
Senior System Software Engineer - Local AI Automation

NVIDIA Corporation • Pune District

On-site
INR 1,500,000 - 2,200,000
Competitive salary
Generous benefits package
Career growth opportunities
Senior Manager, Software Engineering - Employee Productivity
Senior Manager, Software Engineering - Employee Productivity

NVIDIA Corporation • Pune District

On-site
INR 4,000,000 - 7,000,000
Senior Staff Site Reliability Engineer
Senior Staff Site Reliability Engineer

NVIDIA Corporation • Pune District

On-site
INR 3,600,000 - 6,000,000
Senior Developer Relations Manager - FSI
Senior Developer Relations Manager - FSI

NVIDIA Corporation • Mumbai

On-site
INR 3,000,000 - 6,000,000
Senior MLOps Engineer - DSX Enablement
Senior MLOps Engineer - DSX Enablement

NVIDIA Corporation • Pune District

On-site
INR 4,000,000 - 7,000,000
Senior Staff Site Reliability Engineer
Senior Staff Site Reliability Engineer

NVIDIA • India

On-site
INR 4,200,000 - 6,600,000
Senior Manager, Software Engineering - Employee Productivity
Senior Manager, Software Engineering - Employee Productivity

NVIDIA Corporation • Pune District

On-site
INR 3,500,000 - 7,000,000
Senior AI Infrastructure Engineer - DGX Cloud
Senior AI Infrastructure Engineer - DGX Cloud

NVIDIA Corporation • India

On-site
INR 4,500,000 - 7,000,000