Senior Inference Performance Engineer, Agent-Driven Optimization

2100 NVIDIA USA; Santa Clara (CA); Hybrid; USD 124,000 - 242,000; Full time

Job summary

NVIDIA Corporation is seeking a Senior Inference Performance Engineer to push the performance envelope of large-scale AI inference on NVIDIA GPUs. You will architect optimized workflows, profile and tune inference stacks, and drive measurable wins across TensorRT-LLM, vLLM, and related frameworks.

You will collaborate with TensorRT-LLM, SGLang, and GPU teams to translate profiling insights into deployable improvements, including patches, kernels, and deployment recipes.

Qualifications

• BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Math, or a related field, or equivalent experience.
• 3+ years of relevant engineering experience.
• Extensive knowledge of AI model execution efficiency and optimization including batching, throughput-latency tradeoffs, KV cache and memory limits, parallel processing, MoE serving, quantization, and serving SLAs.
• Hands-on benchmarking and profiling GPU workloads using Nsight Systems, Nsight Compute, CUPTI, or PyTorch profiler; interpreting kernel-level data.
• Strong Python engineering skills and the ability to navigate and modify large C++/CUDA serving codebases.
• Rigorous experimental methodology with controlled single-variable comparisons and reproducible benchmarks.
• Strong written and verbal communication skills to explain performance tradeoffs clearly.

Responsibilities

• Distill performance instincts into reusable skills, workflows, and autonomous agent tasks.
• Review agent experiments, validate findings, and curate best-known configurations.
• Improve AI inference workloads to increase throughput-per-GPU and interactivity via configuration, batching, parallelism, KV cache, quantization, and decoding settings.
• Profile workloads across TensorRT-LLM, SGLang, vLLM, Dynamo on NVIDIA GPUs.
• Use Nsight Systems, kernel traces, and internal tools to measure performance.
• Land upstream improvements to serving frameworks, kernels, and deployment recipes while preserving model accuracy.

Skills

Python

C++/CUDA

Nsight Systems

Nsight Compute

PyTorch profiler

Performance optimization

Education

BS/MS/PhD in Computer Science/Engineering or related

Tools

TensorRT-LLM

SGLang

vLLM

CUDA kernels

NCCL/SHMEM

Job description

NVIDIA Corporation is seeking a Senior Inference Performance Engineer to push the performance envelope of large-scale AI inference on NVIDIA GPUs. You will architect optimized workflows, profile and tune inference stacks, and drive measurable wins across TensorRT-LLM, vLLM, and related frameworks.

You will collaborate with TensorRT-LLM, SGLang, and GPU teams to translate profiling insights into deployable improvements, including patches, kernels, and deployment recipes.