Senior AI Inference Systems Engineer

NVIDIA; Toronto; On-site; CAD 170,000 - 275,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking highly skilled software engineers to build AI inference systems that scale across multi-GPU, multi-node, and multi-cloud environments. You will optimize kernels, contribute to vLLM, and push the boundaries of accelerated computing for AI.

You’ll collaborate across inference, compiler, scheduling, and performance teams, conducting research and publishing results while delivering high-performance software for NVIDIA’s AI ecosystem.

Qualifications

• Bachelor's degree in CS/CE/SE with 7+ years of experience, or Master’s with 5+ years, or PhD with top-tier ML Systems publications
• Strong programming in Python and C/C++, Go or Rust a plus, solid CS fundamentals
• Performance engineering for ML frameworks (e.g., PyTorch) and model serving systems (vLLM, SGLang)
• CUDA GPU programming expertise and profiling tools (Nsight)
• Experience with containers/orchestration (Docker, Kubernetes, Slurm) and Linux features

Responsibilities

• Contribute features to vLLM and optimize inference framework with advanced parallelism and GPU features
• Develop, optimize, and benchmark GPU kernels; build high-level DSLs and compiler infrastructure for peak hardware utilization
• Define and build inference benchmarking methodologies; contribute to MLPerf Inference submissions
• Architect scheduling and orchestration of containerized large-scale inference deployments on GPU clusters across clouds
• Conduct and publish original research integrating ideas into NVIDIA software products

Skills

Python

C/C++

Go

Rust

Algorithms & Data Structures

Operating Systems

Computer Architecture

Parallel Programming

Distributed Systems

Deep Learning Theories

Education

Bachelor's degree in CS/CE/SE

Master's degree in CS/CE/SE

PhD in ML Systems / GPU Architecture / HPC

Tools

CUDA

Nsight Systems/Compute

Docker

Kubernetes

Slurm

Linux Namespaces

Job description

NVIDIA is seeking highly skilled software engineers to build AI inference systems that scale across multi-GPU, multi-node, and multi-cloud environments. You will optimize kernels, contribute to vLLM, and push the boundaries of accelerated computing for AI.

You’ll collaborate across inference, compiler, scheduling, and performance teams, conducting research and publishing results while delivering high-performance software for NVIDIA’s AI ecosystem.