Senior System Software Engineer - Dynamo-Triton Inference Server

NVIDIA; United States; On-site; USD 152,000 - 241,500; Full time

Benefits offered by this job

Eligible for equity

Additional benefits

Job summary

A leading technology firm is seeking a Senior System Software Engineer to develop GPU-accelerated AI inference software and contribute to the Triton Inference Server. The role requires 5+ years of experience in deep learning software, strong skills in Rust and C++, and familiarity with distributed systems. You will work in a fast-paced environment, driving customer adoption and building robust, production-ready software. Compensation ranges from $152,000 to $241,500, depending on experience and location.

Qualifications

• 5+ years of professional experience working on deep learning software.
• Experience with high-scale distributed systems and ML systems.
• Strong programming and software design skills including debugging, performance analysis, and test design.

Responsibilities

• Develop world‑class GPU‑accelerated AI inference serving software.
• Contribute to feature development and drive broad customer adoption.
• Drive the convergence of the Triton Inference Server and NVIDIA Dynamo stacks.

Skills

Rust

C++

Python

Distributed systems programming

Communication skills

Education

MS or PhD in Computer Science or relevant field

Tools

TensorRT

PyTorch

ONNX

OpenVINO

vLLM

TRT-LLM

Job description

We are looking for a Senior System Software Engineer to work on the Dynamo‑Triton Inference Server. NVIDIA is hiring software engineers for its GPU‑accelerated deep learning software team. Academic and commercial groups around the world are using GPUs to power a revolution in AI, enabling breakthroughs in image classification, speech recognition, and natural language processing. We are a fast‑paced team building a highly‑performant AI inference platform to make design and deployment of new AI models easier and more accessible to all users.

What You’ll Be Doing

• Develop world‑class GPU‑accelerated AI inference serving software.
• Contribute to feature development and drive broad customer adoption.
• Drive the convergence of the Triton Inference Server and NVIDIA Dynamo stacks to establish a unified, high‑performance inference platform that serves both large language model (LLM) and non‑LLM workloads.
• Be an active member of the open‑source deep learning software engineering community.
• Balance building robust, production‑grade software with optimizing prediction throughput, latency, and adoption of next‑generation inference technologies.

What We Need To See

• MS or PhD in Computer Science or relevant field (or equivalent experience).
• 5+ years of professional experience working on deep learning software.
• Excellent Rust and C++ skills, familiarity with Python, and strong programming and software design skills including debugging, performance analysis, and test design.
• Experience with high‑scale distributed systems and ML systems.
• Strong communication skills and ability to work in a fast‑paced, agile team environment.

Ways To Stand Out From The Crowd

• Prior experience with AI frameworks and engines such as TensorRT, PyTorch, ONNX, OpenVINO, vLLM, or TRT‑LLM.
• Knowledge of GPU memory management, cache management, or high‑performance networking.
• Experience with distributed systems programming.
• Experience contributing to large open‑source projects, including GitHub workflow, bug tracking, branching and merging, OSS licensing, and patch handling.

Compensation & Benefits

• Base salary ranges from $152,000 to $241,500 for Level 3 and $184,000 to $287,500 for Level 4, depending on location and experience.
• Eligible for equity and additional benefits.

NVIDIA uses AI tools in its recruiting processes.

Equal Opportunity Statement

NVIDIA is committed to fostering a diverse work environment and is an equal‑opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law.