Engineering Manager – GPU Deep Learning Inference & Frameworks

Socket.dev; Santa Clara (UT); On-site; USD 224,000 - 357,000; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA is seeking a Manager for Deep Learning Inference Software to lead a world-class engineering team. You will drive strategy and execution for GPU-accelerated inference frameworks, collaborating with diverse teams to deliver optimized end-to-end pipelines.

The role requires 6+ years in software with 3+ in leadership, strong C/C++, Python, and hands-on GPU programming. Expect a competitive salary with equity and a broad benefits package.

Qualifications

• MS/PhD or equivalent in CS/EE or related field.
• 6+ years of software development experience, with 3+ years in technical leadership or engineering management.
• Strong background in C/C++ software design; Python experience is a plus.
• Hands‑on GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
• Proven record of deploying or optimizing deep learning models in production environments.
• Experience leading teams using Agile or collaborative software development practices.

Responsibilities

• Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
• Drive strategy, roadmap, and execution of inference frameworks engineering, focusing on Client AI.
• Partner with compiler, libraries, and research teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
• Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
• Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications.

Skills

C/C++ software design

Python

Leadership

Education

MS/PhD in CS/EE or equivalent

Tools

CUDA

Triton

CUTLASS

NCCL

NVSHMEM

Job description

NVIDIA is seeking a Manager for Deep Learning Inference Software to lead a world-class engineering team. You will drive strategy and execution for GPU-accelerated inference frameworks, collaborating with diverse teams to deliver optimized end-to-end pipelines.

The role requires 6+ years in software with 3+ in leadership, strong C/C++, Python, and hands-on GPU programming. Expect a competitive salary with equity and a broad benefits package.