Engineering Manager: GenAI Inference & Deployment at Scale

NVIDIA Gruppe; Santa Clara (CA); Hybrid; USD 224,000 - 431,000; Full time

Benefits offered by this job

Equity

Job summary

NVIDIA is seeking an Engineering Manager to lead a team deploying and serving Large Language Models (LLMs) and Vision‑Language Models (VLMs) at scale. You will guide a platform crossing model optimization, inference systems, and distributed infrastructure to deliver high‑performance production deployments.

You will mentor engineers, collaborate with researchers, and shape the roadmap for scalable AI inference, optimizing latency, throughput, and cost across GPU platforms.

Qualifications

• Master’s or PhD in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• 8+ years total experience, including 3+ years in management/leadership.
• Hands-on experience with LLMs/VLMs and modern DL architectures.
• Expertise in inference optimization techniques: quantization, speculative decoding, batching, KV-cache.

Responsibilities

• Lead, mentor, and grow a high-performing team building and operating a platform for deploying GenAI models at scale.
• Drive deployment and optimization of LLMs and VLMs for low-latency, high-throughput inference.
• Profile and optimize end-to-end DL workloads across model, inference stack, and GPU hardware.
• Collaborate with research teams to bring new architectures from prototype to production.
• Set technical strategy and roadmaps for model deployment, inference optimization, and reliability.
• Establish engineering standards for benchmarking, deployment, and continuous performance optimization.
• Partner with internal and external teams for seamless GenAI model deployment.

Skills

People management

Leadership

Deep learning

Communication

Education

Master's or PhD in CS/CE

Tools

TensorRT

TensorRT-LLM

vLLM

SGLang

Job description

NVIDIA is seeking an Engineering Manager to lead a team deploying and serving Large Language Models (LLMs) and Vision‑Language Models (VLMs) at scale. You will guide a platform crossing model optimization, inference systems, and distributed infrastructure to deliver high‑performance production deployments.

You will mentor engineers, collaborate with researchers, and shape the roadmap for scalable AI inference, optimizing latency, throughput, and cost across GPU platforms.