Senior Software Engineer - Quantized Inference & ML Precision

NVIDIA Corporation; Santa Clara (CA); On-site; USD 152,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA Corporation in Santa Clara, CA seeks a Senior Software Engineer specializing in Quantized Inference to speed up LLM deployment. You will implement quantized and sparse recipes in inference engines, optimize export pipelines, and build benchmarks for throughput and interactivity across Megatron-LM, vLLM, and related tools.

Key work includes writing Triton kernels, integrating quantize/dequantize paths, and collaborating with cross-team inference groups to push performance.

Qualifications

• Proficient in Python; familiarity with C++
• Strong software engineering fundamentals: concise, well-tested code; fluent with AI-assisted tooling
• Experience with ML accelerators with a basic understanding of how certain ML layers affect execution time
• Familiarity with PyTorch internals (custom ops, autograd, export) or equivalent framework
• MS/PhD in Computer Science or related field, or equivalent experience
• 4+ years in a relevant software engineering role

Responsibilities

• Implement quantized and sparse recipes in inference engines (vLLM, TRT-LLM, SGLang)
• Own model export pipelines (ModelOpt, Megatron-LM <-> HuggingFace)
• Build prototypes and benchmarking harnesses to evaluate recipe throughput/interactivity
• Develop data analysis tooling and visualizations for numerics debugging
• Improve developer productivity across the team: CI, build systems, training infrastructure, pipeline friction
• Participate in code reviews and incorporate feedback

Skills

Python

C++

PyTorch internals

Triton kernels

AI-assisted tooling

Education

MS/PhD in Computer Science or related field

Tools

PyTorch

vLLM

TRT-LLM

SGLang

HuggingFace ModelOpt

Job description

NVIDIA Corporation in Santa Clara, CA seeks a Senior Software Engineer specializing in Quantized Inference to speed up LLM deployment. You will implement quantized and sparse recipes in inference engines, optimize export pipelines, and build benchmarks for throughput and interactivity across Megatron-LM, vLLM, and related tools.

Key work includes writing Triton kernels, integrating quantize/dequantize paths, and collaborating with cross-team inference groups to push performance.