Engineering Manager, Deep Learning Inference & GPU

NVIDIA; Georgia; On-site; USD 224,000 - 431,250; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape software powering today’s AI—from LLMs to multimodal generative AI—accelerated on NVIDIA GPUs.

You’ll guide strategy for OSS inference frameworks, collaborate with compiler and research teams, and drive performance optimization across CUDA ecosystems.

Qualifications

• MS/PhD or equivalent experience in CS/EE or related field.
• 6+ years of software development experience with 3+ years in technical leadership.
• Strong background in C/C++ and Python; GPU programming experience a must.

Responsibilities

• Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
• Guide strategy, roadmap, and execution of OSS inference frameworks engineering.
• Partner with compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines.
• Oversee performance tuning and optimization of large-scale models for LLM and generative AI.
• Guide engineers in CUDA, Triton, CUTLASS, and multi-GPU communications.
• Represent the team in roadmap discussions aligned with NVIDIA’s AI strategy.
• Foster a culture of technical excellence and continuous innovation.

Skills

C/C++ programming

Python

Software architecture

Education

MS/PhD or equivalent in CS/EE or related field

Tools

CUDA

Triton

CUTLASS

NIXL

NCCL

NVSHMEM

Job description

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape software powering today’s AI—from LLMs to multimodal generative AI—accelerated on NVIDIA GPUs.

You’ll guide strategy for OSS inference frameworks, collaborate with compiler and research teams, and drive performance optimization across CUDA ecosystems.