Senior GPU Inference Engineer — TensorRT & Compiler Optimizations

NVIDIA Corporation; Santa Clara (CA); Hybrid; USD 170,000 - 260,000; Full time

Benefits offered by this job

Equity

Job summary

NVIDIA Corporation is seeking talented engineers to join the TensorRT team to advance high-performance AI inference on NVIDIA GPUs. This Senior Software Engineer role focuses on developing GPU inference software, compiler optimizations, and performance improvements for generative AI models.

You will design and implement GPU code generation techniques, profile workloads, and collaborate with hardware and software teams to push limits of inference performance across architectures.

Qualifications

• BS/MS/PhD in CS/CE or equivalent experience.
• 4+ years of software development with substantial codebase.
• Strong C++ skills and experience building performance-critical software.
• Understanding of GPU architecture, memory hierarchies, and parallel programming.
• Excellent problem-solving and communication; able to work independently or across teams.

Responsibilities

• Design and implement efficient GPU code generation in TensorRT balancing performance, compile time, and startup latency.
• Develop compiler optimizations (fusion, dynamic scheduling, specialization for shapes/types/architectures).
• Profile workloads and identify bottlenecks to guide improvements.
• Develop in C++ and Python for tooling, testing, and performance analysis.
• Collaborate with GPU architects, kernel developers, compiler engineers, and DL experts.

Skills

C++ proficiency

GPU programming

Performance analysis

Problem solving

Communication

Independent work

Education

BS/MS/PhD in CS/CE or equivalent

Tools

CUDA

Triton

CuTe DSL

CUTLASS

MLIR

NVIDIA Nsight

PyTorch

ONNX Runtime

TensorRT

JAX

SGLang

Job description

NVIDIA Corporation is seeking talented engineers to join the TensorRT team to advance high-performance AI inference on NVIDIA GPUs. This Senior Software Engineer role focuses on developing GPU inference software, compiler optimizations, and performance improvements for generative AI models.

You will design and implement GPU code generation techniques, profile workloads, and collaborate with hardware and software teams to push limits of inference performance across architectures.