Lead Architect, Scaled AI Inference

Nvidia Corporation in; Santa Clara (CA); On-site; USD 320,000 - 489,000; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA is seeking a Distinguished Engineer to lead the strategy for scaled-out AI inferencing, designing high-throughput, low-latency pipelines and model serving across cloud and data center environments.

You will steer cross-domain optimization, drive architectural decisions, and collaborate with leadership to ensure production-grade availability and performance of NVIDIA’s AI solutions.

Qualifications

• 16+ years in technical roles with AI infrastructure focus.
• 7-10+ years of leadership experience.
• BS/MS or higher in systems/software engineering or related fields.
• Deep technical expertise in GPU, hardware acceleration, and low-level performance tuning (CUDA).

Responsibilities

• Architect distributed pipelines for high-throughput, low-latency inference at massive scale.
• Drive performance tuning at kernel/driver level and optimize GPU resource management.
• Guide open source projects (Dynamo, TensorRT-LLM, vLLM, Ray) for NVIDIA hardware.
• Orchestrate model lifecycles including automated deployment and scaling.
• Collaborate with customers and partners to set industry standards for performance and availability.
• Lead all technical aspects from ideation to deployment in a large scoped program.

Skills

GPU architecture

CUDA/kernel tuning

Cloud-native architectures

Leadership in tech teams

Education

BS/MS or higher in systems/software engineering

Tools

TensorRT-LLM

Dynamo

Kubernetes

Linux

Ray

Job description

NVIDIA is seeking a Distinguished Engineer to lead the strategy for scaled-out AI inferencing, designing high-throughput, low-latency pipelines and model serving across cloud and data center environments.

You will steer cross-domain optimization, drive architectural decisions, and collaborate with leadership to ensure production-grade availability and performance of NVIDIA’s AI solutions.