Senior ML Engineer: Quantized Inference & Pipelines

NVIDIA AI; Redmond (WA); On-site; USD 140,000 - 200,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA AI is seeking an engineer to implement quantized and sparse recipes in inference engines and to manage model export pipelines for correct serialization. You will build benchmarking harnesses and data analysis tools to improve developer productivity through infrastructure and CI improvements.

The role requires strong Python and C++ skills, experience with ML accelerators, and familiarity with PyTorch internals, with 4+ years in software engineering. MS/PhD in CS is preferred.

Qualifications

• Requires 4+ years in a relevant software engineering role.
• Proficiency in Python and familiarity with C++, plus strong fundamentals.

Responsibilities

• Implement quantized and sparse recipes within inference engines and manage model export pipelines to ensure correct serialization.
• Develop benchmarking harnesses and data analysis tools to improve performance and visibility.
• Improve developer productivity through infrastructure and CI improvements.

Skills

Python

C++

Triton Kernels

PyTorch

Quantized Inference

Model Compression

Machine Learning Accelerators

vLLM

TRT-LLM

SGLang

Megatron-LM

ModelOpt

Software Engineering

Data Analysis

Numerical Debugging

Large Language Models

Education

Master's degree in Computer Science

PhD in Computer Science

Tools

PyTorch internals

Job description

NVIDIA AI is seeking an engineer to implement quantized and sparse recipes in inference engines and to manage model export pipelines for correct serialization. You will build benchmarking harnesses and data analysis tools to improve developer productivity through infrastructure and CI improvements.

The role requires strong Python and C++ skills, experience with ML accelerators, and familiarity with PyTorch internals, with 4+ years in software engineering. MS/PhD in CS is preferred.