Senior Math Libraries Engineer - LLM Integration and Developer Experience

NVIDIA; Santa Clara (CA); Hybrid; USD 152,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA seeks an engineer to accelerate the integration of APIs across the CUDA-X math library stack for modern LLM and agentic AI workflows. You will drive API design, documentation, and high‑performance tooling to enable AI developers and human engineers to build powerful applications.

We value a PhD or MSc with 3+ years of experience, deep knowledge of LLMs and code generation, and a track record in designing clear, composable APIs.

Qualifications

• PhD or MSc in Computational Science, CS, Applied Math, or related field (or equivalent experience).
• 3+ years of experience.
• Strong knowledge of LLMs, finetuning, RL, RAGs, MCP, and tooling.
• Experience designing clear, composable APIs and well-documented code.
• Ability to prioritize multiple projects and work independently.
• Excellent collaboration, communication, and documentation habits.

Responsibilities

• Drive architecture and implementation of LLM-first APIs and docs.
• Collaborate with stakeholders to deliver LLM-enhanced library releases.
• Build metrics and tooling to measure impact of AI code generation.
• Prototype tooling to translate math library APIs for code generation.

Skills

LLMs

Fine-tuning

Reinforcement learning

RAGs

Agenting tooling

API design

Education

PhD or MSc in Computational Science or CS

Tools

CUDA-X libraries

cuBLAS

cuFFT

Documentation tools

Job description

We are looking for a passionate and energized engineer to accelerate the integration of APIs across the CUDA-X math library software stack into modern LLM and agentic AI workflows. Leading organizations globally are using GPU-powered data centers for AI, data analytics, and scientific simulations, driving advancements in fields like LLMs, computer vision, CAE, EDA, and autonomous vehicles. Our team develops the GPU-accelerated libraries and SDKs essential for these technologies.

In this role, you will be responsible for ensuring both human engineers using LLMs and AI agents alike build high-performance applications with ease through effective utilization of CUDA-X libraries. Do you have the rare blend of technical, developer experience and communication skills? If so, we would love to learn more about you!

What You'll Be Doing:

• Drive the architecture and implementation of math libraries APIs and documentation structures designed to be LLM-first: introspectable, explainable, and easily synthesized by AI agents.

• Work with internal and external stakeholders to deliver timely LLM-enhanced library releases.

• Build metrics, tools and processes to measure impact and quality of LLM/agentic code generation.

• Prototype tooling and solutions to help transform math libraries’ APIs into LLM-friendly representations across popular codegen tools.
  

What We Need To See:

• PhD or MSc’s degree in Computational Science, Computer Science, Applied Math, or related science or engineering field of study is preferred (or equivalent experience)

• 3+ years of experience

• Robust knowledge of LLMs, finetuning, RL, building RAGs, MCP, and building agenting tooling

• Proven experience designing clear, composable APIs and writing high-quality, well-documented code for complex technical domains

• Ability to prioritize multiple projects and work independently with minimal direction

• Excellent collaboration, communication, and documentation habits
  

Ways To Stand Out From The Crowd:

• Prior work in AI-assisted software engineering, code generation, or programming language design

• Familiarity with CUDA-X math library APIs: cuBLAS, cuFFT, cuSOLVER, cuSPARSE, etc.
  

NVIDIA’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics, and revolutionized parallel computing for science and engineering. More recently, GPU deep learning ignited modern AI — the next era of computing — with the GPU acting as the brain of computers, robots, and self-driving cars that can perceive and understand the world. Today, we are increasingly known as “the AI computing company.” We're looking to grow our company and build our teams with the smartest people in the world! Join us at the forefront of technological advancement. NVIDIA is widely considered to be one of the technology world's most desirable employers. We have some of the most forward-thinking and talented people in the world working for us. If you're creative, autonomous and love a challenge, we want to hear from you!

#LI-Hybrid

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD - 241,500 USD for Level 3, and 184,000 USD - 287,500 USD for Level 4.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until September 19, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.