Senior LLM/VLM Inference Performance Engineer

NVIDIA Corporation (old); Santa Clara (CA); Hybrid; USD 184,000 - 357,000; Full time

Job summary

NVIDIA is seeking a Senior Software Engineer – AI Inference Performance to advance LLM and VLM inference on NVIDIA GPU-accelerated systems. You will analyze workloads, optimize latency, throughput and efficiency, and contribute to open-source inference engines while collaborating with model, framework, kernel, networking, and GPU architecture teams.

You will build speed-of-light and roofline models, tune serving hyperparameters, and develop performance-critical kernels using CUDA, Triton, and

Qualifications

• More than 6 years of experience in full‑stack LLM/VLM inference performance involving models, serving, distributed runtimes, kernels, and hardware.
• Strong Python, Rust and/or C++ skills with hands‑on CUDA or GPU programming experience.
• Deep understanding of GPU architecture, Tensor Cores, memory hierarchy, caches, and occupancy.

Responsibilities

• Lead end-to-end analysis of LLM/VLM inference processes.
• Define representative prefill and decode workloads.
• Optimize latency (time to first token, inter-token), throughput, and KV-cache performance.
• Isolate costs in preprocessing, encoder, and decoder for multimodal models.
• Build and validate performance models and benchmarks.
• Collaborate across teams and contribute upgrades to inference frameworks like TensorRT-LLM, vLLM, or Triton.

Skills

Python

Rust

C++

CUDA

Nsight Systems

Nsight Compute

TensorRT-LLM

vLLM

SGLang

Triton

NCCL

Education

BS or MS in Computer Science or Computer Engineering or related field

Tools

CUDA

CUTLASS

Triton

Job description

NVIDIA is seeking a Senior Software Engineer – AI Inference Performance to advance LLM and VLM inference on NVIDIA GPU-accelerated systems. You will analyze workloads, optimize latency, throughput and efficiency, and contribute to open-source inference engines while collaborating with model, framework, kernel, networking, and GPU architecture teams.

You will build speed-of-light and roofline models, tune serving hyperparameters, and develop performance-critical kernels using CUDA, Triton, and