Senior Edge AI GPU & LLM Inference Engineer

NVIDIA Corporation; Santa Clara (CA); On-site; USD 224,000 - 431,000; Full time

Job summary

NVIDIA Corporation’s Local AI team is building software that optimizes LLM inference on edge AI hardware. You’ll evaluate open-source frameworks, map models to GPU architecture, and drive performance characterization across multi-node configurations.

You will own validation workflows, develop automated inference recipes, and collaborate with partners to address hardware-specific questions and issues.

Qualifications

• BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience.
• 12+ years of software engineering with depth in GPU computing, ML systems, or high-performance inference.
• Strong Python or C++ programming, software design, and software engineering skills.
• Hands-on experience with GPU kernel development or optimization (CUDA/C++, Triton, or equivalent).
• Working knowledge of LLM inference internals: attention mechanisms, KV-cache management, continuous batching, quantization formats, and tensor parallelism.
• Container engineering expertise: multi-architecture Docker or OCI builds, layer optimization, runtime configuration, NVIDIA Container Toolkit.
• Strong analytical skills: ability to form a performance hypothesis, design an experiment, interpret results, and communicate findings clearly.

Responsibilities

• Track and evaluate innovations in leading open-source LLM inference frameworks — identify performance-critical features and algorithmic improvements relevant to NVIDIA edge AI hardware.
• Analyze how new model architectures and inference algorithms map onto NVIDIA GPU architecture — identify mismatch, fallback paths, and optimization opportunities.
• Characterize multi-node inference behavior: collective communication primitives (NCCL/RCCL), topology-aware all-reduce strategies, and parallelism efficiency on edge cluster configurations.
• Produce performance analysis reports mapping theoretical hardware limits to observed inference throughput, latency, and utilization.
• Own the model validation workflow for new model releases: architecture compatibility assessment, inference recipe development, performance characterization, and publication to developer recipe sites.
• Develop and maintain developer-facing inference recipes: keep them accurate as frameworks evolve, automate staleness detection, and build feedback loops from CI results to recipe updates.
• Engage with community and partners on model bring-up questions; serve as the technical point of contact for hardware-specific inference issues related to partner concerns.

Skills

GPU computing

ML systems

High-performance inference

Python / C++

CUDA/Triton kernel dev

LLM inference internals

Container engineering

Analytical skills

Education

BS/MS/PhD in CS/CE/EE

Tools

Docker

OCI builds

NVIDIA Container Toolkit

Job description

NVIDIA Corporation’s Local AI team is building software that optimizes LLM inference on edge AI hardware. You’ll evaluate open-source frameworks, map models to GPU architecture, and drive performance characterization across multi-node configurations.

You will own validation workflows, develop automated inference recipes, and collaborate with partners to address hardware-specific questions and issues.