Senior Software Engineer – TensorRT Edge-LLM

NVIDIA; California (MO); On-site; USD 152,000 - 241,500; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking a software engineer to join its TensorRT Edge-LLM team, focusing on developing a state-of-the-art inference framework for large language models. The role requires a deep understanding of transformer models, modern C++ programming skills, and experience with LLM frameworks. The position offers competitive salary ranges between $152,000 and $287,500, equity, and benefits. NVIDIA values diversity and provides an equal-opportunity workplace.

Qualifications

• 4+ years of relevant software development experience.
• Familiarity with popular LLM frameworks and libraries.
• A track record of strong software design and execution.

Responsibilities

• Develop a state-of-the-art inference framework in modern C++.
• Design and implement optimizations for transformer-based models.
• Benchmark and optimize inference performance across environments.

Skills

Deep understanding of transformer models

Modern C++ programming (C++11/14/17)

Inference optimization techniques

Collaboration across fields

Education

BS, MS, PhD in Computer Science or related field

Tools

TensorRT

CUDA

Job description

Are you passionate about pushing the limits of real-time large language model inference? Join NVIDIA’s TensorRT Edge-LLM team and help shape the next generation of edge AI for automotive and robotics. We build the software stack that enables Large Language, Vision-Language, and Multimodal (LLM/VLM/VLA) models to run efficiently on embedded and edge platforms — delivering cutting‑edge generative AI experiences directly on device.

What You’ll Be Doing

• Develop and evolve a state‑of‑the‑art inference framework in modern C++ that extends TensorRT with autoregressive model serving capabilities, including speculative decoding, LoRA, MoE, and KV cache management.
• Design and implement compiler and runtime optimizations tailored for transformer‑based models running on constrained, real‑time platforms.
• Collaborate with teams across CUDA, kernel libraries, compilers, and robotics to deliver high‑performance, production‑ready solutions.
• Contribute to CUDA kernel and operator development for critical transformer components such as attention, GEMM, and MoE.
• Benchmark, profile, and optimise inference performance across diverse embedded and automotive environments.
• Stay ahead of the rapidly evolving LLM/VLM ecosystem and bring emerging techniques into product‑grade software.

What We Need To See

• BS, MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a closely related field.
• 4+ years of relevant software development experience.
• Deep understanding of transformer models and inference optimisation techniques (for example, quantisation, tensor parallelism, or memory‑efficient scheduling).
• Proficient programming ability with modern C++ (C++11/14/17 and beyond).
• Familiarity with popular LLM frameworks and libraries such as TensorRT, TensorRT‑LLM, vLLM, SGLang, MLC‑LLM, or FlashInfer.
• A track record of strong software design, execution, and collaboration across fields.

Ways To Stand Out From The Crowd

• Demonstrated development experience or open‑source contributions to LLM inference frameworks and libraries, such as SGLang, vLLM, or FlashInfer.
• Proficiency with CUDA, including efficient kernel development, performance profiling, and GPU architecture fundamentals.
• Prior work on autoregressive LLM serving systems, including speculative decoding or KV cache management.
• Familiarity with compiler infrastructure for large language model inference.
• Exposure to robotics or embedded AI pipelines, including optimising for low‑latency, resource‑constrained systems.

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD – 241,500 USD for Level 3, and 184,000 USD – 287,500 USD for Level 4.

You will also be eligible for equity and benefits.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal‑opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, colour, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.