Engineering Manager, LLM Performance

NVIDIA Corporation; Santa Clara (CA); Hybrid; USD 272,000 - 431,000; Full time

Job summary

NVIDIA Corporation in Santa Clara, CA, is seeking an Engineering Manager to lead a team that accelerates LLM/VLM inference across TensorRT LLM, vLLM and SGLang. This hands-on leadership role combines deep technical work with team mentorship to ship high-performance software.

You will work with Researchers and GPU Architects to design and optimize features, drive milestones, and ensure production-grade solutions across datacenter architectures.

Qualifications

• MS/PhD or equivalent in CS/CE/AI.
• 7+ years software engineering, with 3+ years in leadership.
• Proven ability to lead distributed cross-functional teams and deliver production-grade software.

Responsibilities

• Lead and grow a team focused on LLM/VLM inference performance.
• Design, implement, and optimize key performance features for LLM inference.
• Improve inference performance across current and future NVIDIA datacenter GPUs.
• Benchmark workloads with inference teams and tune performance.
• Architect and guide production-ready libraries with developer-friendly APIs.
• Manage project planning, milestones, and cross-functional coordination.

Skills

Leadership & people management

C++

Python

LLM/VLM expertise

CUDA/GPU performance

Education

MS, PhD or equiv in CS/CE/AI

Tools

TensorRT LLM

vLLM

SGLang

Job description

At NVIDIA, we aren't just powering the AI revolution - we're accelerating it. We are accelerating LLM inference across the stack and across all open source LLM frameworks like TensorRT LLM, vLLM and SGLang. With demand for AI exploding, particularly in the realm of large language models (LLMs) and vision language models (VLMs, VLAs), we are significantly expanding our team. We're seeking a highly skilled and driven Engineering Manager to take the lead in accelerating the next generation of LLM/VLM/VLA inference software technologies that will define the future of AI. This is a high-impact, hands-on leadership role at the intersection of deep technical expertise and world-class management. You won't just manage; you'll architect and guide a brilliant team of engineers who are pushing the performance of LLM inference. Your work will be highly collaborative, interfacing directly with NVIDIA Researchers, GPU Architects, and other teams across the company to ensure we ship production-grade, lightning-fast software that sets the global standard for AI performance.

What You'll Be Doing:

• Lead and grow a team responsible for pushing the performance of LLM inference across multiple LLM frameworks, including TensorRT LLM, vLLM, SGLang and Dynamo on our datacenter products.
• Drive the design, implementation and optimization of features that are key to performance in LLM inference.
• Continuously improve the performance of LLM inference on current and upcoming NVIDIA datacenter architectures and GPUs.
• Continuously improve the performance of LLM inference of important foundation models.
• Work with inference benchmark teams to help tune performance for key workloads.
• Integrating cutting-edge technologies developed at NVIDIA and offering an intuitive developer experience for LLM deployment.
• Lead software development execution, with responsibility for project planning, milestone delivery, and cross-functional coordination.

What We Need to See:

• MS, PhD, or equivalent experience in Computer Science, Computer Engineering, AI, or a related technical field.
• 7+ overall years of overall software engineering experience, including 3+ years of technical leadership experience.
• Proven ability to lead and scale high-performing engineering teams, especially across distributed and cross-functional groups.
• Strong background in C++ or Python, with expertise in software design and delivering production-quality software libraries.
• Demonstrated expertise in large language models (LLM) and/or vision language models (VLM) and/or inference in general.

Ways to Stand Out from the Crowd:

• Deep understanding of GPU architecture, CUDA programming, and system-level performance tuning.
• Background in LLM inference or working with frameworks such as TensorRT-LLM, vLLM, or SGLang.
• Passion for building scalable, user-friendly APIs and enabling developers in the AI ecosystem.
• Have a proven track record of growing and managing a team that encourages idea sharing, empowers team members, and provides opportunities for professional growth.

#LI-Hybrid

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD for Level 3, and 272,000 USD - 431,250 USD for Level 4. You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until August 9, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

NVIDIA pioneered accelerated computing. Today, our AI infrastructure powers global intelligence, transforming every industry.

Learn more about NVIDIA.