Engineering Manager - Deep Learning Inference on GPUs

NVIDIA; Massachusetts; On-site; USD 224,000 - 431,000; Full time

Benefits offered by this job

Equity compensation

Comprehensive benefits

Job summary

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. The role focuses on shaping software powering today’s AI systems — from large language models to multimodal generative AI — all accelerated on NVIDIA GPUs.

The Deep Learning Inference team develops and optimizes open-source frameworks such as SGLang, vLLM, and FlashInfer, enabling scalable, efficient AI deployment.

Qualifications

• MS/PhD or equivalent in CS/EE or related field.
• 6+ years in software development.
• 3+ years in technical leadership or engineering management.
• Strong background in C/C++ software design.
• Python proficiency a plus.
• Hands-on GPU programming (CUDA/Triton/CUTLASS).
• Experience deploying or optimizing DL models in production.
• Leadership with Agile or collaborative practices.

Responsibilities

• Lead and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
• Define strategy, roadmap, and execution for OSS inference frameworks engineering.
• Collaborate with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines.
• Oversee performance tuning and optimization of large-scale models for LLM, multimodal, and generative AI applications.
• Guide engineers in CUDA, Triton, CUTLASS and multi-GPU communications (NIXL, NCCL, NVSHMEM).
• Represent the team in roadmap discussions and align with broader NVIDIA strategies.
• Foster a culture of technical excellence and continuous innovation.

Skills

C/C++ design

Python

CUDA

Triton

CUTLASS

Agile

Production deployment

Education

MS/PhD or equivalent

Tools

SGLang

vLLM

FlashInfer

Job description

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing the state of AI model deployment. The role focuses on shaping software powering today’s AI systems — from large language models to multimodal generative AI — all accelerated on NVIDIA GPUs.

The Deep Learning Inference team develops and optimizes open-source frameworks such as SGLang, vLLM, and FlashInfer, enabling scalable, efficient AI deployment.