Senior Inference Solutions Architect: Kubernetes GPU Pipelines

NVIDIA Corporation; Santa Clara (CA); On-site; USD 152,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA Corporation is hiring a Solutions Architect focused on AI inference to design and deploy scalable inference pipelines on Kubernetes and GPU clusters. You will collaborate with engineering and customer teams to deliver enterprise AI solutions and production-grade inference systems.

You will optimize models with Dynamo/Triton/TensorRT-LLM, supervise GPU allocation, and guide disaggregated inference deployments, while mentoring peers through complex issues and deployments.

Qualifications

• 5+ years of experience as a Solutions Architect or in a similar role deploying distributed AI inference workloads on Kubernetes.
• Experience with model optimization and serving using one of NVIDIA Dynamo, Triton Inference Server, or TensorRT-LLM.
• Knowledge of GPU allocation, memory hierarchies (HBM, DRAM, SSD), and low-latency networking (RDMA, UCX).

Responsibilities

• Build inference pipelines with tools like NVIDIA Dynamo and distribute tasks among GPU workers.
• Collaborate with DevOps to orchestrate disaggregated inference using Kubernetes for complex workloads.
• Accelerate inference pipelines using TensorRT-LLM, vLLM, SGLang and integrate with disaggregated inference backends.
• Provide mentorship and technical leadership to customers and internal teams during deployments.

Skills

Distributed systems

Low-latency inference

Mentorship

Customer collaboration

Education

Bachelor's degree in CS/Engineering

Tools

Kubernetes

NVIDIA Dynamo

Triton Inference Server

TensorRT-LLM

NIM Operator

vLLM

KServe

SGLang

Job description

NVIDIA Corporation is hiring a Solutions Architect focused on AI inference to design and deploy scalable inference pipelines on Kubernetes and GPU clusters. You will collaborate with engineering and customer teams to deliver enterprise AI solutions and production-grade inference systems.

You will optimize models with Dynamo/Triton/TensorRT-LLM, supervise GPU allocation, and guide disaggregated inference deployments, while mentoring peers through complex issues and deployments.