Senior AI Inference Platform PM — Performance & Scale

NVIDIA; Santa Clara (CA); On-site; USD 208,000 - 328,000; Full time

Job summary

NVIDIA is seeking a highly technical Product Manager to own AI inference optimization on NVIDIA hardware, from single-GPU workstations to large data centers. You will translate deep optimization techniques into capabilities that customers can adopt across the inference stack.

The role spans framework integration with TensorRT-LLM, vLLM, SGLang, and NVIDIA Dynamo, plus benchmarking, release readiness, and cross-functional collaboration to deliver measurable performance gains and compelling

Qualifications

• 12+ years in product management at a tech company or comparable senior leadership experience.
• Deep expertise in AI inference optimization: caching, quantization, speculative decoding, disaggregated serving.
• Familiarity with inference/orchestration frameworks: TensorRT-LLM, vLLM, SGLang, NVIDIA Dynamo.
• Proven ability to work independently from ambiguous problem spaces to shipped outcomes.
• Operational experience with live product releases, quality/regression processes, and customer issues.
• Ability to translate low-level capabilities into business value (lower TCO, faster responses, better GPU utilization).
• BS/MS/PhD in CS/CE or equivalent experience.

Responsibilities

• Own the inference performance roadmap; define model representation, memory/state management, scheduling, and token generation.
• Build platforms with generalization across models, topologies, and customer sizes for easy adoption.
• Define performance strategy for agentic workloads and multi-turn sessions.
• Shape framework integration across TensorRT-LLM, vLLM, SGLang, and Dynamo; partner with open-source and internal teams.
• Own benchmarking, measurement methods, and credible performance claims.
• Manage day-to-day product operations: release readiness, quality bars, regression tracking, and customer feedback loops.

Skills

AI inference optimization

Product management leadership

Independent decision-making

Cross-functional collaboration

Operational excellence

Communication

Education

BS/MS/PhD in Computer Science or Computer Engineering

Tools

TensorRT-LLM

vLLM

SGLang

Dynamo

Job description

NVIDIA is seeking a highly technical Product Manager to own AI inference optimization on NVIDIA hardware, from single-GPU workstations to large data centers. You will translate deep optimization techniques into capabilities that customers can adopt across the inference stack.

The role spans framework integration with TensorRT-LLM, vLLM, SGLang, and NVIDIA Dynamo, plus benchmarking, release readiness, and cross-functional collaboration to deliver measurable performance gains and compelling