GPU Inference Engineer — Deep Learning

2100 NVIDIA USA; California (MO); On-site; USD 124,000 - 241,500; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA seeks a Software Engineer specializing in Deep Learning Inference to design, build and optimize GPU-accelerated software powering AI applications. You will contribute to high-performance inference frameworks, including vLLM, SGLang, and related NVIDIA libraries, across datacenter and edge hardware.

You’ll work with the DL community to implement the latest algorithms for public release, focusing on performance improvements for state-of-the-art LLMs and Generative AI on NVIDIA accelerators.

Qualifications

• Pursuing or recently completed a MS or PhD in Computer Engineering, Computer Science, EECS, AI or related field.
• Strong software development experience.
• Excellent C/C++ programming and software design skills; Python is a plus.
• Experience with training, deploying or optimizing DL model inference is advantageous.

Responsibilities

• Performance optimization, analysis, and tuning of DL models across domains like LLMs and Generative AI.
• Scale DL model performance across architectures and NVIDIA accelerators.
• Contribute features to inference libraries and open-source frameworks.
• Collaborate with cross-functional teams across frameworks and NVIDIA libraries.

Skills

C/C++

Software design

Python

Agile methodologies

Education

MS or PhD in CS/EECS/AI

Tools

CUDA

CUTLASS

OAI Triton

NCCL

vLLM

SGLang

Job description

NVIDIA seeks a Software Engineer specializing in Deep Learning Inference to design, build and optimize GPU-accelerated software powering AI applications. You will contribute to high-performance inference frameworks, including vLLM, SGLang, and related NVIDIA libraries, across datacenter and edge hardware.

You’ll work with the DL community to implement the latest algorithms for public release, focusing on performance improvements for state-of-the-art LLMs and Generative AI on NVIDIA accelerators.