Lead Architect, Scaled AI Inference Systems

NVIDIA; Santa Clara (CA); On-site; USD 320,000 - 489,000; Full time

Job summary

NVIDIA is seeking a senior leader to define and drive the global strategy for scaled-out AI inference across enterprise and cloud environments.

You will architect high-throughput, low-latency distributed pipelines and model serving strategies, guiding deployment, versioning, and intelligent scaling.

This role demands 16+ years in technical roles, 7–10+ years in leadership, and deep GPU/CUDA expertise, with a track record of delivering complex, scalable solutions.

Qualifications

• 16+ years in technical roles with a focus on AI infrastructure
• 7-10+ years of leadership experience
• BS/MS or higher or equivalent in systems/software engineering
• Deep technical expertise in GPU architecture and CUDA
• Proven success delivering complex scalable solutions
• Strong technical leadership across cross-functional teams
• Excellent communication and collaboration with customers and partners

Responsibilities

• Architect distributed pipelines for high-throughput, low-latency inference
• Co-optimize hardware-software and tune performance at kernel/driver level
• Guide open source projects (Dynamo, TensorRT-LLM, vLLM, SGLang, Kubernetes)
• Lead full lifecycle model management: deployment, versioning, scaling
• Engage with customers, infrastructure providers, and partners
• Oversee ideation to deployment and ongoing operations

Skills

AI infrastructure

leadership

GPU architecture

CUDA

cloud-native

Education

BS/MS or higher in systems/software engineering

Tools

TensorRT-LLM

Kubernetes

Linux

Job description

NVIDIA is seeking a senior leader to define and drive the global strategy for scaled-out AI inference across enterprise and cloud environments.

You will architect high-throughput, low-latency distributed pipelines and model serving strategies, guiding deployment, versioning, and intelligent scaling.

This role demands 16+ years in technical roles, 7–10+ years in leadership, and deep GPU/CUDA expertise, with a track record of delivering complex, scalable solutions.