Senior AI Inference Performance Engineer — Equity Eligible

Nvidia Corporation in; Santa Clara (CA); On-site; USD 184,000 - 357,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking a Senior Software Engineer to advance AI inference performance on GPU-accelerated systems. You will optimize LLM/VLM workloads, profile with Nsight tools, and contribute to open-source inference engines while collaborating with model, kernel, and networking teams.

The role emphasizes end-to-end analysis, batching, KV-cache management, and quantization, with a focus on lower latency, higher throughput, and scalable deployments across NVIDIA platforms.

Qualifications

• BS or MS in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• Strong programming skills in Python, C++, and/or Rust; hands-on CUDA or GPU programming experience.
• Experience with speed-of-light analysis, roofline models, microbenchmarks, profiling tools.

Responsibilities

• Lead end-to-end analysis of LLM/VLM inference processes and optimize latency, throughput, and KV cache usage.
• Build speed-of-light and roofline models to quantify performance headroom across GPUs.
• Profile workloads using Nsight Systems/Compute, PyTorch Profiler, and custom instrumentation to remove bottlenecks.
• Tune serving hyperparameters and techniques (batching, caching, quantization, speculative decoding, CUDA Graphs).
• Develop performance-critical kernels in CUDA, CUTLASS, Triton, or related technologies.
• Establish benchmarks, regressions gates, and collaborate across teams to upgrade inference software (TensorRT-LLM, vLLM, SGLang).

Skills

Python

C++

Rust

CUDA

GPU programming

Education

BS or MS in CS/CE or related field

Tools

Nsight Systems

Nsight Compute

Job description

NVIDIA is seeking a Senior Software Engineer to advance AI inference performance on GPU-accelerated systems. You will optimize LLM/VLM workloads, profile with Nsight tools, and contribute to open-source inference engines while collaborating with model, kernel, and networking teams.

The role emphasizes end-to-end analysis, batching, KV-cache management, and quantization, with a focus on lower latency, higher throughput, and scalable deployments across NVIDIA platforms.