Senior GPU AI Inference Engineer

Nvidia Corporation; Santa Clara (CA); Hybrid; USD 224,000 - 431,250; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is hiring a Senior System Software Engineer to develop open source software for GPU-accelerated AI model inference and to advance Dynamo-supported inference engines. The role emphasizes performance, scalability, and integration with stateful, multi-turn agent workloads.

Strong Rust/Python skills, PhD or Master’s with 10+ years experience, and familiarity with modern LLM APIs are required. Hybrid work options apply, with equity and benefits to accompany base compensation.

Qualifications

• Masters or PhD or equivalent experience.
• 10+ years in Computer Science, Computer Engineering, or related field.
• Ability to work in a fast-paced, agile team environment.
• Excellent Rust/Python programming and software design skills, including debugging, performance analysis, and test design.

Responsibilities

• Develop open source software to serve inference of trained AI models running on GPUs.
• Contribute to the development of disaggregated serving for Dynamo-supported inference engines and expand capabilities for agentic workloads.
• Innovate in inference-state management to reduce latency and improve GPU utilization.
• Build and evolve Dynamo's distributed inference frontend across vLLM, SGLang, and TensorRT-LLM.

Skills

Rust programming

Python programming

Performance analysis

LLM semantics

Education

Master's or PhD in CS/CE or related

Job description

NVIDIA is hiring a Senior System Software Engineer to develop open source software for GPU-accelerated AI model inference and to advance Dynamo-supported inference engines. The role emphasizes performance, scalability, and integration with stateful, multi-turn agent workloads.

Strong Rust/Python skills, PhD or Master’s with 10+ years experience, and familiarity with modern LLM APIs are required. Hybrid work options apply, with equity and benefits to accompany base compensation.