Senior LLM Inference Architect — Performance & Benchmarking

NVIDIA Gruppe; Santa Clara (CA); On-site; USD 184,000 - 357,000; Full time

Benefits offered by this job

Equity and benefits

Job summary

NVIDIA in Santa Clara is seeking a seasoned software professional to characterize workloads for the latest LLMs and inference servers. You will join forces with the performance marketing team to craft engaging content and showcase NVIDIA's inference achievements, while collaborating with AI startup engineers to set benchmarking standards.

You will help build a dynamic inference data results site and develop end-to-end profiling tools to keep pace with Generative AI advancements, contributing to

Qualifications

• Master's or PhD in Computer Science, Computer Engineering, or related fields, or equivalent experience.
• 6+ years of software development experience.
• Deep learning inference serving, PyTorch programming, profiling, and compiler optimizations.
• Experience developing client-server LLM applications with OpenAI API or MCP and identifying performance bottlenecks.
• Solid understanding of CPU and GPU microarchitecture and performance characteristics.
• Experience with complex software projects like frameworks, compilers, or operating systems.
• Proficiency with AI coding agents like Claude Code, Codex, Cursor.
• Excellent written and verbal communication skills and ability to work independently and collaboratively in a fast-paced environment.

Responsibilities

• Characterize workload of LLMs and inference servers to maintain leadership.
• Collaborate with performance marketing to create content and updates.
• Work with engineers from AI startups to establish benchmarking methodologies.
• Develop an evolving inference performance data results website.
• Create E2E profiling and analysis tools to keep pace with Generative AI.
• Contribute to PyTorch, TRT-LLM, vLLM, and SGLang to drive AI advancements.
• Verify that new GPU launches deliver industry-leading performance.
• Guide inference serving direction across software, research, and product teams.
• Utilize the latest coding agents and inference tech to improve team efficiency.

Skills

DL inference serving

PyTorch

Profiling

Compiler optimizations

OpenAI API

MCP

CPU/GPU microarchitecture

LLM apps

AI agents

Communication

Education

Master's or PhD in CS/CE

Tools

Claude Code

Codex

Cursor

Job description

NVIDIA in Santa Clara is seeking a seasoned software professional to characterize workloads for the latest LLMs and inference servers. You will join forces with the performance marketing team to craft engaging content and showcase NVIDIA's inference achievements, while collaborating with AI startup engineers to set benchmarking standards.

You will help build a dynamic inference data results site and develop end-to-end profiling tools to keep pace with Generative AI advancements, contributing to