Senior Deep Learning Software Engineer, Inference and Model Optimization

NVIDIA Corporation; Santa Clara, Northern (CA, KY); Hybrid; USD 184,000 - 357,000; Full time

Benefits offered by this job

Equity

Comprehensive benefits

Competitive salary

Job summary

NVIDIA is seeking a Senior Deep Learning Software Engineer in Santa Clara to develop and scale automated inference and deployment for generative AI models such as LLMs and diffusion models. You will push performance with tensor and sequence parallelism while integrating optimized kernels in a modular platform.

You will collaborate across NVIDIA to advance inference software like TRT, TRT-LLM, and TRTModelOptimizer, while driving broad model support and adoption with excellent user experience.

Qualifications

• Masters, PhD, or equivalent in CS, AI, or related field.
• 5+ years of relevant work in Deep Learning.
• Excellent software design, debugging, performance analysis, and test design.
• Strong Python and PyTorch skills; experience with ML tooling (HuggingFace).
• Strong algorithms and programming fundamentals.
• Excellent written and verbal communication; ability to work independently and with teams.

Responsibilities

• Train, develop, and deploy state‑of‑the generative AI models like LLMs and diffusion models using NVIDIA's AI software stack.
• Leverage TorchDynamo, torch.export, torch.compile to analyze and extract model graph representations for automated deployment.
• Develop high‑performance optimization techniques for inference (tensor/pipeline parallelism, efficient attention kernels, kv‑caching).
• Collaborate with teams across NVIDIA to implement performant kernel designs in deployment solutions.
• Profile GPU kernel performance to identify hardware/software optimization opportunities.
• Innovate on inference performance to maintain leadership of TRT, TRT‑LLM, TRTModelOptimizer.
• Architect and design a modular, scalable software platform with broad model support and adoption.

Skills

Python

PyTorch

HuggingFace

Deep Learning

Education

Masters or PhD in CS/AI

Tools

CUDA

CUTLASS

Triton

TorchDynamo

Job description

NVIDIA is at the forefront of the generative AI revolution! The Algorithmic Model Optimization Team specifically focuses on optimizing generative AI models such as large language models (LLM) and diffusion models for maximal inference efficiency using techniques ranging from neural architecture search and pruning to sparsity, quantization, and automated deployment strategies. Our work includes conducting applied research to improve model efficiency as well as developing an innovative software platform (TRTModelOptimizer). Our software is used both internally across NVIDIA and externally by research and engineering teams alike developing best‑in‑class AI models.

We are now looking for a Senior Deep Learning Software Engineer to develop and scale up our automated inference and deployment solution.

What you’ll be doing

• Train, develop, and deploy state‑of‑the generative AI models like LLMs and diffusion models using NVIDIA's AI software stack.
• Leverage and build upon the torch2.0 ecosystem (TorchDynamo, torch.export, torch.compile, etc…) to analyze and extract standardized model graph representation from arbitrary torch models for our automated deployment solution.
• Develop high‑performance optimization techniques for inference, such as automated model sharding techniques (e.g. tensor parallelism, sequence parallelism), efficient attention kernels with kv‑caching, and more.
• Collaborate with teams across NVIDIA to use performant kernel implementations within our automated deployment solution.
• Analyze and profile GPU kernel‑level performance to identify hardware and software optimization opportunities.
• Continuously innovate on the inference performance to ensure NVIDIA's inference software solutions (TRT, TRT‑LLM, TRTModelOptimizer) can maintain and increase its leadership in the market.
• Play a pivotal role in architecting and designing a modular and scalable software platform to provide an excellent user experience with broad model support and optimization techniques to increase adoption.

What we need to see

• Masters, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related field.
• 5+ years of relevant work or research experience in Deep Learning.
• Excellent software design skills, including debugging, performance analysis, and test design.
• Strong proficiency in Python, PyTorch, and related ML tools (e.g. HuggingFace).
• Strong algorithms and programming fundamentals.
• Good written and verbal communication skills and the ability to work independently and collaboratively in a fast‑paced environment.

Ways to stand out from the crowd

• Contributions to PyTorch, JAX, or other Machine Learning Frameworks.
• Knowledge of GPU architecture and compilation stack, and capability of understanding and debugging end‑to‑end performance.
• Familiarity with NVIDIA's deep learning SDKs such as TensorRT.
• Prior experience in writing high‑performance GPU kernels for machine learning workloads in frameworks such as CUDA, CUTLASS, or Triton.

Compensation & Benefits

• Highly competitive salaries and a comprehensive benefits package.
• Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.
• The base salary range is 184,000 USD - 287,500 USD for Level4, and 224,000 USD - 356,500 USD for Level5.
• You will also be eligible for equity and benefits.

Application Deadline

Applications for this job will be accepted at least until August28,2026.

Diversity & Inclusion

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer.

As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA pioneers accelerated computing. Today, our AI infrastructure powers global intelligence, transforming every industry.

Learn more about NVIDIA.