Senior Deep Learning Inference Engineer (GPU-Optimized AI)

NVIDIA; New York (NY); On-site; USD 152,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA seeks a Senior Software Engineer specializing in Deep Learning Inference for our growing team in New York. You will design, build, and optimize GPU-accelerated software powering today’s AI applications, contributing to high-performance open-source frameworks and model serving pipelines.

You’ll implement latest algorithms for public release, improve performance for LLM and Generative AI models across NVIDIA accelerators, and leverage CUDA, Triton, CUTLASS, NCCL and related tools to

Qualifications

• Masters or PhD or equivalent experience in relevant field (Computer Engineering, Computer Science, EECS, AI).
• 5+ years of relevant software development experience.
• Excellent C/C++ programming and software design skills. SW Agile skills are helpful and Python experience is a plus.
• Prior experience with training, deploying or optimizing the inference of DL models in production is a plus.
• Prior background with performance modeling, profiling, debug, and code optimization or architectural knowledge of CPU and GPU is a plus.
• GPU programming experience (CUDA, OAI TRITON or CUTLASS) is a plus.

Responsibilities

• Performance optimization, analysis, and tuning of DL models in various domains like LLM, Multimodal and Generative AI.
• Scale performance of DL models across different architectures and types of NVIDIA accelerators.
• Contribute features and code to NVIDIA’s inference libraries, vLLM and SGLang, FlashInfer and LLM software solutions.
• Work with cross-collaborative teams across frameworks, NVIDIA libraries and inference optimization innovative solutions.

Skills

C/C++ programming

Python experience

Software design

Agile experience

Education

Masters or PhD in Computer Science/Engineering/EECS or equivalent

Tools

CUDA

CUTLASS

OAI Triton

NCCL

NVSHMEM

vLLM

SGLang

Job description

NVIDIA seeks a Senior Software Engineer specializing in Deep Learning Inference for our growing team in New York. You will design, build, and optimize GPU-accelerated software powering today’s AI applications, contributing to high-performance open-source frameworks and model serving pipelines.

You’ll implement latest algorithms for public release, improve performance for LLM and Generative AI models across NVIDIA accelerators, and leverage CUDA, Triton, CUTLASS, NCCL and related tools to