Engineering Manager: GPU-Accelerated AI Inference

NVIDIA; Illinois; On-site; USD 224,000 - 431,250; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software to lead a world-class engineering team advancing AI model deployment on NVIDIA GPUs. You will shape OSS inference frameworks like SGLang, vLLM, and FlashInfer, guiding strategy, roadmap, and performance tuning for LLMs and multimodal models across datacenters and edge devices.

This role demands 6+ years in software development with 3+ years in leadership, a strong C/C++ background, GPU programming expertise, and a proven

Qualifications

• MS/PhD or equivalent in CS/EE or related field.
• 6+ years of software development experience with 3+ years in leadership.
• Strong C/C++ background; Python a plus.
• Hands-on GPU programming and performance optimization.
• Proven track record deploying models in production.
• Experience with Agile or collaborative development.

Responsibilities

• Lead, mentor, and scale a high-performing inference software team.
• Define strategy, roadmap, and execution for OSS inference frameworks.
• Collaborate with compiler, libraries, and research teams to optimize pipelines.
• Oversee performance tuning for large-scale models (LLM, multimodal).
• Advise on CUDA, Triton, CUTLASS, and multi-GPU communication practices.

Skills

C/C++ design

Python

GPU programming

Agile leadership

Leadership/mentoring

Education

MS/PhD or equivalent

Tools

CUDA

Triton

CUTLASS

NCCL

NVSHMEM

Job description

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software to lead a world-class engineering team advancing AI model deployment on NVIDIA GPUs. You will shape OSS inference frameworks like SGLang, vLLM, and FlashInfer, guiding strategy, roadmap, and performance tuning for LLMs and multimodal models across datacenters and edge devices.

This role demands 6+ years in software development with 3+ years in leadership, a strong C/C++ background, GPU programming expertise, and a proven