Performance DL Software Engineer (Inference)

NVIDIA; United States; On-site; USD 124,000 - 196,000; Full time

Benefits offered by this job

Equity

Job summary

NVIDIA is seeking a Deep Learning Software Engineer focused on TensorRT performance to analyze and improve the inference ecosystem. You will contribute to optimization and benchmarking across TensorRT, Torch-TensorRT, and related tools, collaborating with teams across AI, automotive, and robotics.

The role requires strong C++/Python, experience with DL frameworks, and a proven ability to drive performance improvements. Base salary varies by level and location, with equity and benefits included.

Qualifications

• 2 years of relevant software development experience.
• Strong C++ and Python programming skills.
• Experience with DL frameworks (e.g. PyTorch, JAX, TensorFlow, ONNX) and inference libraries (e.g. TensorRT, TensorRT-LLM, vLLM, SGLang, FlashInfer).
• Experience with performance analysis and performance optimization.

Responsibilities

• Establish groundbreaking performance benchmarking methodologies and analysis workflows and identify performance issues and opportunities for NVIDIA's inference ecosystem (e.g. TensorRT/TensorRT-EdgeLLM/Torch-TensorRT).
• Contribute features and code to NVIDIA/OSS inference frameworks including but not limited to TensorRT/TensorRT-EdgeLLM/Torch-TensorRT.
• Develop new model pipelines for NVIDIA's inference ecosystem with optimized performance including but not limited to areas like quantization, scheduling, memory management, and distributed inference to set the gold standard for Gen AI performance.
• Work with cross-collaborative teams inside and outside of NVIDIA across generative AI, automotive, robotics, image understanding, and speech understanding to set directions and develop innovative inference solutions.
• Scale performance of deep learning models across different architectures and types of NVIDIA accelerators.

Skills

C++

Python

Performance analysis

Deep learning concepts

Education

Bachelor's degree

Master's degree

PhD

Tools

TensorRT

TensorRT-LLM

Torch-TensorRT

vLLM

SGLang

FlashInfer

CUDA

PyTorch

JAX

TensorFlow

ONNX

TileIR

CuTeDSL

Cutlass

Triton

Job description

NVIDIA is seeking a Deep Learning Software Engineer focused on TensorRT performance to analyze and improve the inference ecosystem. You will contribute to optimization and benchmarking across TensorRT, Torch-TensorRT, and related tools, collaborating with teams across AI, automotive, and robotics.

The role requires strong C++/Python, experience with DL frameworks, and a proven ability to drive performance improvements. Base salary varies by level and location, with equity and benefits included.