Senior Software Engineer – TensorRT Edge-LLM

NVIDIA; Deutschland; Vor Ort; EUR 159.000 - 248.000; Vollzeit

Zusammenfassung

NVIDIA seeks an experienced engineer to advance an inference framework in C++ that extends TensorRT with autoregressive model serving, including speculative decoding and KV cache management. You will optimize compilers, runtimes, and CUDA kernels for transformer workloads on embedded and automotive platforms.

Join a collaborative team across CUDA, kernel libraries, and robotics to deliver production-grade, high-performance software.

Qualifikationen

• BS/MS/PhD or equivalent in CS/EE or related field.
• 4+ years of relevant software development experience.
• Deep understanding of transformer models and inference optimization techniques.

Aufgaben

• Develop and evolve a state-of-the-art inference framework in C++ extending TensorRT with autoregressive model serving.
• Design and implement compiler and runtime optimizations for transformer models on constrained real-time platforms.
• Collaborate with CUDA, kernel libraries, compilers, and robotics teams to deliver high-performance solutions.
• Contribute to CUDA kernel and operator development for attention, GEMM, MoE.
• Benchmark, profile, and optimize inference performance across embedded and automotive environments.
• Stay ahead of evolving LLM/VLM ecosystem and translate techniques into product-grade software.

Kenntnisse

Modern C++

Transformer models

Inference optimization

CUDA

TensorRT

KV cache management

Ausbildung

BS/MS/PhD or equivalent in CS/EE

Tools

TensorRT-LLM

vLLM

SGLang

MLC-LLM

FlashInfer

CUDA toolkit

Jobbeschreibung

What you7ll be doing:

• Develop and evolve a state-of-the-art inference framework in modern C++ that extends TensorRT with autoregressive model serving capabilities, including speculative decoding, LoRA, MoE, and KV cache management.
• Design and implement compiler and runtime optimizations tailored for transformer-based models running on constrained, real-time platforms.
• Collaborate with teams across CUDA, kernel libraries, compilers, and robotics to deliver high-performance, production-ready solutions.
• Contribute to CUDA kernel and operator development for critical transformer components such as attention, GEMM, and MoE.
• Benchmark, profile, and optimize inference performance across diverse embedded and automotive environments.
• Stay ahead of the rapidly evolving LLM/VLM ecosystem and bring emerging techniques into product-grade software.

What we need to see:

• BS, MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a closely related field.
• 4+ years of relevant software development experience.
• Deep understanding of transformer models and inference optimization techniques (e.g., quantization, tensor parallelism, or memory-efficient scheduling).
• Proficient programming ability with modern C++ (C++11/14/17 and beyond).
• Familiarity with popular LLM frameworks and libraries such as TensorRT, TensorRT-LLM, vLLM, SGLang, MLC-LLM, or FlashInfer.
• A track record of strong software design, execution, and collaboration across fields.

Ways to stand out from the crowd:

• Demonstrated development experience or open-source contributions to LLM inference frameworks and libraries, such as SGLang, vLLM, or FlashInfer.
• Proficiency with CUDA, including efficient kernel development, performance profiling, and GPU architecture fundamentals.
• Prior work on autoregressive LLM serving systems, including speculative decoding or KV cache management.
• Familiarity with compiler infrastructure for large language model inference.
• Exposure to robotics or embedded AI pipelines, including optimizing for low-latency, resource-constrained systems.

NVIDIA is widely considered to be one of the technology world7s most desirable employers. We hire some of the most brilliant and forward-thinking people in the world. If you thrive on innovation, autonomy, and technical excellence, come join us to shape the future of edge AI.

#LI-Hybrid

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD - 241,500 USD for Level 3, and 184,000 USD - 287,500 USD for Level 4.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until August 9, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.