Senior Quantized Inference Engineer - Accelerate LLMs

NVIDIA AI; Redmond (WA); On-site; USD 184,000 - 287,500; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking a Senior Software Engineer for Quantized Inference to accelerate the development of efficient inference recipes for LLMs. You will implement quantized and sparse recipes, work on kernel and model-level implementations, and collaborate with partner teams to optimize throughput and interactivity across Megatron-LM, ModelOpt, and vLLM.

The role requires strong Python skills with familiarity in C++, experience with PyTorch internals, and 4+ years in software engineering.

Qualifications

• 4+ years in a relevant software engineering role.
• MS/PhD in Computer Science or related field or equivalent experience.
• Familiarity with PyTorch internals (custom ops, autograd, export) or equivalent framework.
• Experience with ML accelerators and quantization concepts is a plus.

Responsibilities

• Implement quantized and sparse recipes in inference engines (vLLM, TRT-LLM, SGLang).
• Own model export pipelines (ModelOpt, Megatron-LM HuggingFace).
• Build prototypes and benchmarking harnesses for throughput and interactivity.
• Develop data analysis tooling and visualizations for numerics debugging.
• Improve developer productivity: CI, build systems, training infrastructure.
• Participate in code reviews and incorporate feedback.

Skills

Python

C++

PyTorch internals

ML accelerators

Education

MS/PhD in Computer Science or related field

Job description

NVIDIA is seeking a Senior Software Engineer for Quantized Inference to accelerate the development of efficient inference recipes for LLMs. You will implement quantized and sparse recipes, work on kernel and model-level implementations, and collaborate with partner teams to optimize throughput and interactivity across Megatron-LM, ModelOpt, and vLLM.

The role requires strong Python skills with familiarity in C++, experience with PyTorch internals, and 4+ years in software engineering.