Engineering Manager, GPU AI Inference & Open-Source

Nvidia Corporation; Santa Clara (CA); On-site; USD 272,000 - 431,250; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape software powering AI systems—LLMs, multimodal, generative AI—accelerated on NVIDIA GPUs.

Lead strategy for OSS inference frameworks, guide cross-team collaborations, and optimize large-scale models for production. Equity eligibility and a comprehensive benefits package are included.

Qualifications

• MS/PhD or equivalent in CS, ECE, or related field.
• 6+ years software development experience with 3+ years in leadership.
• Strong background in C/C++ and GPU programming.
• Hands-on experience deploying or optimizing DL models in production.
• Experience leading teams using Agile practices.

Responsibilities

• Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
• Guide the strategy, roadmap, and execution of NVIDIA's OSS inference frameworks engineering.
• Partner with internal compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
• Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
• Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications (NIXL, NCCL, NVSHMEM).
• Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA's broader AI and software strategies.
• Foster a culture of technical excellence, open collaboration, and continuous innovation.

Skills

C/C++ software design

Python (plus)

Leadership / mentoring

Agile / collaborative development

GPU programming (CUDA, Triton, CUTLASS

Performance optimization

Education

MS/PhD or equivalent in CS/EE or related field

Tools

CUDA

Triton

CUTLASS

Job description

NVIDIA is seeking an exceptional Manager, Deep Learning Inference Software, to lead a world-class engineering team advancing AI model deployment. You will shape software powering AI systems—LLMs, multimodal, generative AI—accelerated on NVIDIA GPUs.

Lead strategy for OSS inference frameworks, guide cross-team collaborations, and optimize large-scale models for production. Equity eligibility and a comprehensive benefits package are included.