Senior Product Manager - AI Platform Inference

NVIDIA; Santa Clara (CA); On-site; USD 208,000 - 328,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking a Senior Product Manager for AI Platform Inference to build tools, SDKs, and libraries that enable developers to deploy inference on NVIDIA GPUs. You will craft product strategy, roadmaps, and go-to-market plans while partnering with developers to shape model optimization software.

The role requires strong knowledge of inference deployment and performance optimization, a BS/MS in CS/CE (or equivalent), and 6+ years in technical product management.

Qualifications

• Experience with inference deployment and optimization software (e.g., TensorRT-LLM, Triton, TorchAO).
• Strong understanding of GenAI or ML concepts, especially performance optimization.
• BS or MS in CS/CE or equivalent experience, 6+ years in technical product management.
• Excellent communication and collaboration skills.

Responsibilities

• Create products to help developers build better Inference deployments.
• Develop product strategy, roadmaps, and go-to-market plans.
• Collaborate with developers to build roadmaps for model optimization software.
• Work with leadership to align with company strategy.

Skills

Inference deployment

Performance optimization

Product management

Communication

Education

BS/MS in CS/CE or equivalent

Tools

TensorRT-LLM

Triton

TorchAO

SGLang

Job description

Inference is the fastest growing and most competitive area in Generative AI today. It is where AI models impact our daily life, and where ever bit of accuracy and performance matters for quality, safety, and cost. Inference is also constantly evolving, with new acceleration algorithms, usecases, and deployment techniques. As a Senior Product Manager for AI Platform Inference you will be responsible for building the tools, SDKs, and libraries which enables developers' Inference deployments to thrive on NVIDIA GPUs.

What You'll Be Doing

• Create products to help developers build better Inference deployments
• Develop product strategy, roadmaps, and go-to-market plans
• Collaborate with internal and external developers to build product-based roadmaps for model optimization software
• Work with leadership to align with and drive company strategy

What We Need To See

• Experience with Inference deployment and optimization software (ex. vLLM, SGLang, FlashInfer, TensorRT-LLM, Triton, Dynamo, TorchAO, etc.)
• Demonstrable knowledge of GenAI or machine learning concepts, particularly around performance optimization, and software development and delivery
• BS or MS degree in Computer Science, Computer Engineering, or similar experience (or equivalent experience)
• 6+ years of technical product management, or similar, experience at a technology company
• Strong communication and interpersonal skills

Ways To Stand Out From The Crowd

• Experience leading optimization products for Inference
• Working on Open Source & Github-first developer products with deep customer interactions
• Knowledge of GPU architecture, HW/SW co-design, and performance profiling

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 168,000 USD - 258,750 USD for Level 4, and 208,000 USD - 327,750 USD for Level 5.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until August 9, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

JR2022724