Senior AI Inference Systems Engineer (GPU, Open Source)

NVIDIA Corporation; Santa Clara (CA); Hybrid; USD 224,000 - 431,000; Full time

Benefits offered by this job

Equity

Employee benefits

Job summary

NVIDIA Corporation is seeking a Senior System Software Engineer to advance Dynamo, its GPU-accelerated deep learning inference platform. You will develop open-source software for AI model inference, expand disaggregated serving, and optimize latency and throughput across NVIDIA's distributed inference stack.

Ideal candidates have a PhD or equivalent, strong Rust/Python skills, and experience with LLM API semantics. This role is hybrid in the Santa Clara area, with equity and benefits offered.

Qualifications

• Masters or PhD or equivalent experience in CS/Engineering
• Strong Rust and Python programming abilities with performance analysis
• Experience in open-source AI inference frameworks and high-performance software design

Responsibilities

• Develop open-source software to serve inference of trained AI models on GPUs
• Contribute to disaggregated serving for Dynamo inference engines and extend to agentic workloads
• Balance performance, latency, and throughput across distributed inference frontend components

Skills

Rust

Python

Performance analysis

Debugging

Education

Advanced CS degree

Tools

vLLM

TensorRT-LLM

SGLang

Job description

NVIDIA Corporation is seeking a Senior System Software Engineer to advance Dynamo, its GPU-accelerated deep learning inference platform. You will develop open-source software for AI model inference, expand disaggregated serving, and optimize latency and throughput across NVIDIA's distributed inference stack.

Ideal candidates have a PhD or equivalent, strong Rust/Python skills, and experience with LLM API semantics. This role is hybrid in the Santa Clara area, with equity and benefits offered.