Senior Architect, Scaled AI Inference

NVIDIA; United States; On-site; USD 200,000 - 320,000; Full time

Benefits offered by this job

Benefits package

Job summary

NVIDIA is seeking a senior leader to shape the global strategy for scaled-out AI inferencing. You will architect high-throughput, low-latency distributed pipelines and model serving strategies for massive scale and reliability across enterprise and cloud environments.

You will collaborate with NVIDIA leadership to define the technical roadmap, drive full-lifecycle deployment, automated scaling, and orchestration layers enabling world-class AI models to run efficiently on NVIDIA hardware.

Qualifications

• 16+ years in technical roles with a recent focus on AI infrastructure and large-scale inference.
• 7–10+ years of leadership experience in engineering teams.
• BS/MS or higher in systems/software engineering or related fields.
• Deep expertise in GPU architecture, hardware acceleration, and CUDA.
• Proven ability to deliver complex, transparent production systems with measurable performance.

Responsibilities

• Architect distributed pipelines for high-throughput, low-latency AI inference at scale.
• Lead cross-domain optimization across hardware and software for production models.
• Guide open-source collaborations (e.g., Dynamo, TensorRT-LLM) to NVIDIA products.
• Oversee full lifecycle of model deployment, versioning, and automatic scaling.

Skills

GPU architecture

Performance tuning

Distributed systems

Leadership

Kernel-level optimization

Education

BS/MS or higher in systems/software engineering

Tools

CUDA

Kubernetes

Linux

TensorRT-LLM

vLLM

Ray

Job description

NVIDIA is seeking a senior leader to shape the global strategy for scaled-out AI inferencing. You will architect high-throughput, low-latency distributed pipelines and model serving strategies for massive scale and reliability across enterprise and cloud environments.

You will collaborate with NVIDIA leadership to define the technical roadmap, drive full-lifecycle deployment, automated scaling, and orchestration layers enabling world-class AI models to run efficiently on NVIDIA hardware.