Senior AI Inference Performance Engineer

NVIDIA Corporation; Santa Clara (CA); On-site; USD 184,000 - 357,000; Full time

Benefits offered by this job

Benefits package

Equity

Job summary

NVIDIA is seeking a Senior Software Engineer – AI Inference Performance to advance LLM and VLM inference on GPU-accelerated systems. You will lead end-to-end analysis, define workloads, and push latency and throughput improvements across models, serving software, and distributed runtimes.

The role requires hands-on coding in Python/C++/Rust, deep GPU architecture knowledge, and experience profiling with Nsight tools.

Qualifications

• 6+ years in full-stack LLM/VLM inference performance.
• Strong Python, Rust and/or C++ with GPU profiling experience.
• Experience with Nsight tools and profiling workflows.
• Deep understanding of GPU architecture and memory hierarchy.
• Experience optimizing inference servers and deployment.

Responsibilities

• Lead end-to-end analysis of LLM/VLM inference processes.
• Define representative prefill and decode workloads.
• Optimize latency, throughput, and KV-cache usage.
• Profile workloads using Nsight Systems, Nsight Compute, PyTorch Profiler, and custom tools.
• Eliminate bottlenecks in host code, CUDA kernels, memory, and scheduling.
• Tune batching, KV-cache management, quantization, and model parallelism.

Skills

Python

Rust

C++

Performance analysis

Education

BS/MS in CS/CE

Tools

CUDA

Nsight Systems

Nsight Compute

PyTorch Profiler

Triton

CUTLASS

TensorRT-LLM

vLLM

NCCL

CUDA Graphs

Job description

NVIDIA is seeking a Senior Software Engineer – AI Inference Performance to advance LLM and VLM inference on GPU-accelerated systems. You will lead end-to-end analysis, define workloads, and push latency and throughput improvements across models, serving software, and distributed runtimes.

The role requires hands-on coding in Python/C++/Rust, deep GPU architecture knowledge, and experience profiling with Nsight tools.