Senior AI Inference Systems Engineer (GPU & HPC)

NVIDIA; Santa Clara (CA); Hybrid; USD 184,000 - 357,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking highly skilled software engineers to build AI inference systems that scale across multi-GPU, multi-node, and multi-cloud deployments. You will architect high-performance inference stacks, optimize kernels and compilers, and collaborate across inference, compiler, scheduling, and performance teams to push the frontier of accelerated AI computing.

Responsibilities include contributing features to vLLM, benchmarking, and driving MLPerf Inference submissions, with emphasis on

Qualifications

• Bachelor’s degree in CS/CE/SE with 7+ years of experience or Master’s with 5+ years, or PhD with publications in ML Systems/HPC.
• Strong programming in Python and C/C++; Go or Rust is a plus; solid CS fundamentals.
• Knowledge of performance engineering in ML frameworks (PyTorch) and inference engines (vLLM/SGLang).
• Familiarity with GPU programming (CUDA), memory hierarchies, Nsight tools; experience with profiling.

Responsibilities

• Contribute features to vLLM and optimize the inference framework for NVIDIA GPUs.
• Develop, optimize, and benchmark GPU kernels; extend compiler infrastructure and DSLs.
• Define benchmarking methodologies; contribute to MLPerf Inference submissions.
• Architect containerized large-scale inference deployments across clouds.
• Publish original research and bridge ideas into NVIDIA software products.

Skills

Python

C/C++

Go or Rust

Algorithms & data structures

Distributed systems

GPU programming

CUDA

Performance tuning

PyTorch

NCCL

Education

Bachelor's degree in CS/CE/SE

Master's degree in CS/CE/SE

PhD in ML Systems / GPU architecture / HPC

Tools

Docker

Kubernetes

Slurm

Nsight Systems/Compute

MLIR/LLVM

Triton

TorchDynamo/Inductor

CUTLASS

Job description

NVIDIA is seeking highly skilled software engineers to build AI inference systems that scale across multi-GPU, multi-node, and multi-cloud deployments. You will architect high-performance inference stacks, optimize kernels and compilers, and collaborate across inference, compiler, scheduling, and performance teams to push the frontier of accelerated AI computing.

Responsibilities include contributing features to vLLM, benchmarking, and driving MLPerf Inference submissions, with emphasis on