Senior AI Inference Performance Engineer

NVIDIA; Santa Clara (CA); On-site; USD 184,000 - 357,000; Full time

Job summary

NVIDIA seeks a Senior Software Engineer – AI Inference Performance to push LLM/VLM workloads toward practical performance limits on NVIDIA GPUs. You will lead end-to-end analysis, build performance models, and optimize latency, throughput, and energy efficiency across models, serving software, and distributed runtimes.

The role spans profiling, tuning, and developing high-performance kernels with CUDA, Triton, and CUTLASS while collaborating with model, kernel, networking, and GPU architecture

Qualifications

• 6+ years of experience in full-stack LLM/VLM inference performance.
• Strong programming skills in Python, Rust and/or C++, with CUDA experience.
• Expertise in speed-of-light analysis, roofline models, microbenchmarks, and profiling tools.

Responsibilities

• Lead end-to-end analysis of LLM/VLM inference processes and optimize latency, throughput, and KV-cache.
• Build performance models and translate profiling data into actionable optimizations.
• Profile workloads with Nsight Systems, Nsight Compute, and PyTorch Profiler; remove bottlenecks in host, CUDA kernels, memory, and scheduling.
• Tune serving parameters (batching, KV-cache management, quantization, speculative decoding, CUDA Graphs, model parallelism).
• Develop performance-critical kernels (attention, matrix multiply, mixture-of-experts routing) using CUDA/CUTLASS/Triton.
• Establish benchmarks, run records, and regression gates; collaborate across teams to upgrade TensorRT-LLM, vLLM, SGLang, etc.

Skills

Python

C++

Rust

CUDA

Education

BS or MS in CS/CE or related field

Tools

NVIDIA Nsight Systems

Nsight Compute

PyTorch Profiler

CUDA

Job description

NVIDIA seeks a Senior Software Engineer – AI Inference Performance to push LLM/VLM workloads toward practical performance limits on NVIDIA GPUs. You will lead end-to-end analysis, build performance models, and optimize latency, throughput, and energy efficiency across models, serving software, and distributed runtimes.

The role spans profiling, tuning, and developing high-performance kernels with CUDA, Triton, and CUTLASS while collaborating with model, kernel, networking, and GPU architecture