Senior LLM Inference: GPU Kernel Optimization

NVIDIA; Austin (TX); On-site; USD 184,000 - 287,500; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is seeking a Sr. Inference Engineer to push LLM inference performance through GPU kernel optimization.

You will help develop silicon-measured benchmarking, model-level performance projection tooling, and agentic optimization systems, collaborating across compiler, hardware, and framework teams to surface bottlenecks and deliver measurable gains. You will work on GPU kernel microbenchmarking, end-to-end model performance analysis, and agentic optimization, shaping production inference

Qualifications

• Master's or PhD in CS/CE or related field, or equivalent experience.
• 6+ years of relevant industry experience.
• Experience building or directing agentic AI systems — code generation, automated optimization, or multi-step reasoning workflows.
• Strong Python and C++ skills with proven software engineering fundamentals.
• Hands-on GPU profiling with CUPTI, NSYS, and NCU; attribution of bottlenecks across kernel execution, compiler decisions, and runtime scheduling.
• Direct experience with LLM inference frameworks such as TRT-LLM, SGLang, or vLLM and understanding of how kernel-driven throughput and latency.
• Working knowledge of GPU kernel optimization — CUDA, CUTLASS, Triton; ability to read PTX or SASS output.

Responsibilities

• Develop silicon-measured kernel benchmarking infrastructure.
• Build model-level performance projection tooling.
• Create agentic optimization systems to improve GPU kernels at the assembly level.
• Drive GPU kernel microbenchmarking across configuration space.
• Perform end-to-end model performance analysis and derive optimization policies.
• Collaborate with compiler, hardware, kernel, and framework teams to deliver production-grade gains.

Skills

Python

C++

GPU profiling

LLM frameworks (TRT-LLM, SGLang, vLLM)

Kernel optimization (CUDA, CUTLASS, Tr

Agentic AI systems

Multi-agent orchestration

Education

Master's or PhD in Computer Science/Computer Engineering or related field

Equivalent experience

Tools

CUPTI

NSYS

NCU

Job description

NVIDIA is seeking a Sr. Inference Engineer to push LLM inference performance through GPU kernel optimization.

You will help develop silicon-measured benchmarking, model-level performance projection tooling, and agentic optimization systems, collaborating across compiler, hardware, and framework teams to surface bottlenecks and deliver measurable gains. You will work on GPU kernel microbenchmarking, end-to-end model performance analysis, and agentic optimization, shaping production inference