Senior AI Inference & Kernel Engineer

NVIDIA; Durham (NC); On-site; USD 184,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA in Durham, NC is seeking outstanding AI systems engineers to develop groundbreaking inference software. You will build libraries, code generators, and GPU kernel technologies for NVIDIA's hardware, designing new abstractions and high-performance runtimes for large language models and AI workloads.

You will collaborate across teams and contribute to open source projects, while leveraging a Master's (PhD preferred) and 6+ years of ML/DL experience, with strong Python/C++ and CUDA/Triton

Qualifications

• Master's degree in CS, EE, or related field (PhD preferred).
• 6+ years of ML/DL systems development experience.
• Strong experience with DL frameworks (PyTorch, JAX, TensorFlow, ONNX) and inference engines like vLLM, SGLang, ML C.
• Strong Python and C/C++ programming skills.
• Experience in GPU kernel development and CUDA/CuTile/Triton; matrix multiplication focus.

Responsibilities

• Innovate and develop AI systems technologies for efficient inference.
• Design and optimize kernels for high-impact AI workloads.
• Create extensible abstractions for LLM serving engines.
• Build just-in-time domain-specific compilers and runtimes.
• Collaborate with NVIDIA teams across frameworks, libraries, kernels, and GPU arch groups.
• Contribute to open source projects like FlashInfer, vLLM, and SGLang.

Skills

Python

C/C++

CUDA

GPU kernel development

Education

Master's degree in Computer Science or Electrical Engineering

PhD preferred

Tools

PyTorch

JAX

TensorFlow

ONNX

vLLM

SGLang

M L C

Triton

Job description

NVIDIA in Durham, NC is seeking outstanding AI systems engineers to develop groundbreaking inference software. You will build libraries, code generators, and GPU kernel technologies for NVIDIA's hardware, designing new abstractions and high-performance runtimes for large language models and AI workloads.

You will collaborate across teams and contribute to open source projects, while leveraging a Master's (PhD preferred) and 6+ years of ML/DL experience, with strong Python/C++ and CUDA/Triton