Lead Architect, Scaled AI Inference

Nvidia Corporation; Santa Clara (CA); On-site; USD 320,000 - 489,000; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA is seeking a senior AI infrastructure leader with extensive experience in large-scale inference orchestration and secure, highly available distributed systems. You will steer cross-domain optimization across hardware and software, guiding multi-tenant deployments and cloud/datacenter strategies.

Lead architecture and lifecycle management while engaging with customers, partners, and executive leadership to ensure NVIDIA’s solutions set industry standards for performance and reliability.

Qualifications

• 16+ years in technical roles with focus on AI infrastructure and large-scale inference orchestration
• 7-10+ years of leadership experience
• BS/MS or higher in systems / software engineering or related fields
• Deep technical expertise in GPU architecture, hardware acceleration, and low-level performance tuning (CUDA, kernels)
• Proven success delivering high-impact, transparent resource/utilization and performance insights
• Technical leadership and cross-functional alignment at senior levels
• Strong communication and teamwork to engage engineering, customers, and partners

Responsibilities

• Architect distributed pipelines for high-throughput, low-latency inference systems
• Drive hardware-software co-optimization and performance tuning
• Guide open source projects and ecosystem initiatives (Dynamo, TensorRT-LLM, vLLM, SGLang, Ray)
• Lead model lifecycle management including automated deployment and scaling
• Collaborate with customers and partners to define industry-leading standards
• Oversee the full software and system lifecycle from ideation to evolution

Skills

GPU architecture

Hardware acceleration

Low-level tuning

Cloud-native architecture

Leadership

Cross-functional collaboration

Strategic thinking

Education

BS/MS or higher in systems/ software engineering

Tools

CUDA

Kernels

Linux

Kubernetes

Ray

Job description

NVIDIA is seeking a senior AI infrastructure leader with extensive experience in large-scale inference orchestration and secure, highly available distributed systems. You will steer cross-domain optimization across hardware and software, guiding multi-tenant deployments and cloud/datacenter strategies.

Lead architecture and lifecycle management while engaging with customers, partners, and executive leadership to ensure NVIDIA’s solutions set industry standards for performance and reliability.