Engineering Manager - GPU AI Inference & Frameworks

NVIDIA; Town of Texas (WI); On-site; USD 184,000 - 357,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking a Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape the software powering state-of-the-art AI systems — from large language models to multimodal generative AI — all accelerated on NVIDIA GPUs.

The Deep Learning Inference group develops and optimizes open-source frameworks to make AI deployment scalable, efficient, and accessible — including vLLM / SGLang, and FlashInfer.

Qualifications

• MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or related field.
• 6+ years of software development experience, including 3+ years in technical leadership or engineering management.
• Strong background in C/C++ software design and development; proficiency in Python is a plus.
• Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
• Proven record of deploying or optimizing deep learning models in production environments.
• Experience leading teams using Agile or collaborative software development practices.

Responsibilities

• Lead, mentor, and scale a high-performing engineering team.
• Drive the strategy, roadmap, and execution of inference frameworks engineering focused on Client AI.
• Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines.
• Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
• Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications.
• Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA’s broader AI and software strategies.
• Foster a culture of technical excellence, open collaboration, and continuous innovation.

Skills

C/C++ software design

Python

Leadership / engineering management

Agile software development

Open-source contributions

Education

MS or PhD in Computer Science or Electrical/Computer Engineering

Tools

CUDA

Triton

CUTLASS

NCCL

Job description

NVIDIA is seeking a Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape the software powering state-of-the-art AI systems — from large language models to multimodal generative AI — all accelerated on NVIDIA GPUs.

The Deep Learning Inference group develops and optimizes open-source frameworks to make AI deployment scalable, efficient, and accessible — including vLLM / SGLang, and FlashInfer.