Senior AI Inference Systems Engineer

NVIDIA Corporation; Toronto; Hybrid; CAD 170,000 - 275,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA Corporation's Toronto office seeks highly skilled software engineers to build AI inference systems, optimize GPU kernels, and scale multi‑GPU workloads across clouds. You will contribute features to vLLM, profile performance, and collaborate with inference, compiler, and performance teams to push the frontier of accelerated computing for AI.

The role requires 7+ years (or 5+ years with a Master’s) in CS/CE/SE, strong Python and C/C++, CUDA experience, and familiarity with

Qualifications

• Bachelor’s degree or equivalent experience with 7+ years, or Master’s with 5+ years, or PhD with relevant publications.
• Strong programming in Python and C/C++; Go or Rust a plus; solid CS fundamentals.
• Experience with ML frameworks (PyTorch) and model serving (vLLM/SGLang) preferred.

Responsibilities

• Contribute features to vLLM and optimize inference frameworks for latest NVIDIA GPUs.
• Develop, optimize, and benchmark GPU kernels and compiler tech; build DSLs and compiler infra.
• Define and run inference benchmarks; publish and contribute to MLPerf Inference.
• Architect scheduling/orchestration of containerized large-scale deployments on GPU clusters.
• Conduct original ML systems research and integrate it into NVIDIA products.

Skills

Python

C/C++

Go

Rust

Performance engineering

CUDA

Nsight

Linux

Parallel programming

Distributed systems

Profiling

Docker

Kubernetes

Slurm

MLIR/LLVM

Education

Bachelor's degree in CS/CE/SE

Master's degree in CS/CE/SE

PhD in ML Systems / GPU architecture / HPC

Tools

Docker

Kubernetes

Slurm

Nsight Systems/Compute

CUDA Toolkit

MLIR/LLVM

Job description

NVIDIA Corporation's Toronto office seeks highly skilled software engineers to build AI inference systems, optimize GPU kernels, and scale multi‑GPU workloads across clouds. You will contribute features to vLLM, profile performance, and collaborate with inference, compiler, and performance teams to push the frontier of accelerated computing for AI.

The role requires 7+ years (or 5+ years with a Master’s) in CS/CE/SE, strong Python and C/C++, CUDA experience, and familiarity with