Senior Architect, Scaled AI Inference & Orchestration

NVIDIA; California (MO); On-site; USD 320,000 - 489,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking a leader to drive the global strategy for scaled-out AI inferencing. You will architect high-throughput, low-latency distributed pipelines and model serving strategies for massive scale and reliability across enterprise and cloud environments.

You will define the technical roadmap for full lifecycle deployment, versioning, and automated scaling, collaborating with leadership to ensure peak efficiency on NVIDIA hardware.

Qualifications

• 16+ years in technical roles with a recent long-term focus on AI infrastructure and large-scale inference orchestration.
• 7-10+ years of leadership experience.
• BS/MS or higher or equivalent experience in systems / software engineering, or related engineering fields.
• Deep Technical Expertise: CUDA, GPUs, hardware acceleration, and cloud-native multi-tenant model serving.
• Proven success delivering high-impact technically complex solutions with transparent resource utilization and performance insights.
• Technical Leadership: align leadership and cross-functional teams toward architecture and execution.

Responsibilities

• Architect distributed pipelines for high-throughput, low-latency inference across massive-scale workloads.
• Lead hardware-software co-optimization and kernel-level performance tuning for production-grade model serving.
• Influence open source projects (Dynamo, TensorRT-LLM, vLLM, Ray) to run on NVIDIA hardware.
• Orchestrate model lifecycles including deployment, versioning, and intelligent scaling across cloud and data center environments.
• Engage with customers, infrastructure providers, and partners to set industry standards for performance and availability.
• Oversee end-to-end software and system lifecycle from ideation to evolution.

Skills

Leadership

GPU architecture

CUDA

Cloud-native

Performance tuning

Distributed systems

Education

BS/MS or higher in systems/software engineering

Tools

Kubernetes

TensorRT

Ray

Job description

NVIDIA is seeking a leader to drive the global strategy for scaled-out AI inferencing. You will architect high-throughput, low-latency distributed pipelines and model serving strategies for massive scale and reliability across enterprise and cloud environments.

You will define the technical roadmap for full lifecycle deployment, versioning, and automated scaling, collaborating with leadership to ensure peak efficiency on NVIDIA hardware.