Senior Software Engineer - Local AI

NVIDIA; Austin (TX); On-site; USD 152,000 - 287,500; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA seeks a Senior Systems Software Engineer to tackle client-side AI challenges on Windows and Linux PCs with limited resources. You will collaborate across software, research, architecture, and product teams to advance AI on RTX and DGX platforms, optimizing inference runtimes and memory usage.

Ideal candidates have strong C++ skills, 5+ years in AI inference pipelines, and experience with ONNX RT, PyTorch, TensorRT, llama.cpp, and vLLM. Equity and benefits are part of the package.

Qualifications

• Bachelor's, Master's, or PhD in Computer Science, Software Engineering, Mathematics, or a related field (or equivalent experience).
• Excellent C++ programming and debugging skills with strong data structures and algorithms knowledge.
• 5+ years of experience with proficiency in AI inferencing pipelines and applications using ML/DL frameworks including ONNX RT, PyTorch, Tensor RT, llama.cpp and vLLM.
• Strong analytical and problem-solving abilities, with the ability to multitask in a dynamic environment.
• Outstanding written and oral communication skills enabling effective collaboration with management and engineering teams.

Responsibilities

• Partner with NVIDIA software, research, architecture, and product teams to align strategies and technical needs for fostering the ecosystem of AI on RTX and DGX PCs.
• Collaborate with industry partners to advance AI across graphics, web browsers, and edge devices by driving innovation in open and closed source technologies with system-level support.
• Improve performance on current and next-generation GPU architectures by analyzing and optimizing AI models, data processing pipelines, and inference runtimes.
• Identify, evaluate, and implement compute and memory optimization techniques—such as quantization, distillation, and pruning—for large AI models; fine-tune and compress models for edge devices.

Skills

C++ programming

data structures

problem solving

communication

AI inferencing

Education

Bachelor's/Master's/PhD in CS/SE/Math

Tools

ONNX RT

PyTorch

TensorRT

llama.cpp

vLLM

Job description

NVIDIA has continuously reinvented itself for over two decades. Our invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI - the new era of computing, positioning GPUs as the driving force behind intelligent applications in productivity, gaming, and creative fields, solidifying NVIDIA's position as the leading "AI computing company."

There is a growing emphasis on running AI models locally, closer to the source of data. This approach reduces latency, improves real-time processing, and addresses privacy concerns by minimizing the need for sending data to centralized servers. As technology continues to advance, we can expect client-side AI (local execution) to play a pivotal role in crafting the digital landscape.

Local AI seeks a Senior Systems Software Engineer interested in solving client-side AI challenges on Windows and Linux PCs with limited resources.

What You’ll Be Doing

• Partnering with NVIDIA software, research, architecture, and product teams to align strategies and technical needs for fostering the ecosystem of AI on RTX and DGX PCs.
• Collaborate closely with industry partners to advance AI across critical domains—including graphics, web browsers, and edge devices—by driving innovation in both open and closed source technologies with emphasis on system level support.
• Improving performance on current and next-generation GPU architectures by conducting in-depth analysis and end-to-end optimization of AI models, data processing pipelines, and inference runtime features.
• Identifying, evaluating, and implementing compute and memory optimization techniques—such as quantization, distillation, and pruning—for large AI models; fine-tuning and compressing models to fit edge devices.

What We Need To See

• Bachelor's, Master's, or PhD in Computer Science, Software Engineering, Mathematics, or a related field (or equivalent experience).
• Excellent C++ programming and debugging skills with a strong understanding of data structures and algorithms.
• 5+ years of experience with proficiency in AI inferencing pipelines and applications using ML/DL frameworks, including ONNX RT, PyTorch, Tensor RT, llama.cpp and vLLM.
• Strong analytical and problem-solving abilities, with the ability to multitask effectively in a dynamic environment.
• Outstanding written and oral communication skills enabling effective collaboration with management and engineering teams.

Ways To Stand Out From The Crowd

• Understanding modern techniques in Machine Learning, Deep Neural Networks, and Generative AI with relevant contributions to major open-source projects will be a plus.
• Consistent track record of delivering end-to-end products with geographically distributed teams in multinational product companies.
• Proficiency in lower-level system/GPU programming, CUDA, developing high-performance systems.
• Hands-on experience with building applications using APIs like ONNX RT, DirectX, PyTorch, TensorRT, Vulkan, llama.cpp.

We\'re a top employer known for innovation and growth. We are an equal-opportunity employer and value diversity at our company. With competitive salaries and a generous benefits package, we are widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us and, due to unprecedented growth, our exclusive engineering teams are rapidly growing. If you\'re a creative and autonomous engineer with a real passion for technology, we would like to hear from you.

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD - 241,500 USD for Level 3, and 184,000 USD - 287,500 USD for Level 4.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until August 1, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

, , JR2021826