Inference Performance Engineer — Autonomous Optimization Lead

Thomas To; Santa Clara (CA); On-site; USD 124,000 - 242,000; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA seeks a Senior Inference Performance Engineer to push the performance limits of large-scale AI inference benchmarks. You will optimize AI model execution, harness continuous batching, and explore advanced quantization and memory strategies across TensorRT-LLM, vLLM, and related frameworks.

You will profile and tune GPU workloads with Nsight Systems, Nsight Compute, CUPTI, and PyTorch profiler, delivering measurable headroom while maintaining strict model correctness.

Qualifications

• Advanced knowledge of AI model execution efficiency, including continuous batching, throughput-latency tradeoffs, KV cache constraints, parallel processing, MoE serving, quantization, and serving SLAs.

Responsibilities

• Optimize AI inference workloads to increase throughput per GPU and interactivity by exploring configuration options, batching, parallelism, and memory management.
• Profile workloads with Nsight Systems, Nsight Compute, CUPTI, and PyTorch profiler to identify headroom and drive measured improvements.
• Collaborate with TensorRT-LLM, SGLang, vLLM, and GPU teams to translate profiling insights into performance gains.
• Contribute upstream patches and deployment recipes that improve public performance while ensuring model correctness.

Skills

Python programming

C++/CUDA

Nsight profiling

Education

BS/MS/PhD in CS/CE/EE or related

Tools

Nsight Systems

Nsight Compute

CUPTI

PyTorch profiler

Job description

NVIDIA seeks a Senior Inference Performance Engineer to push the performance limits of large-scale AI inference benchmarks. You will optimize AI model execution, harness continuous batching, and explore advanced quantization and memory strategies across TensorRT-LLM, vLLM, and related frameworks.

You will profile and tune GPU workloads with Nsight Systems, Nsight Compute, CUPTI, and PyTorch profiler, delivering measurable headroom while maintaining strict model correctness.