LLM Training Performance Engineer (GPU Optimization)

NVIDIA Corporation; Town of Montana (WI); On-site; USD 124,000 - 242,000; Full time

Job summary

NVIDIA Corporation seeks an experienced High-Performance LLM Training Engineer to optimize its AI training workloads across frameworks like PyTorch and JAX for thousands of GPUs. You will profile, tune, and implement production-ready software across the deep learning stack and help shape future GPU architectures.

Required are advanced degrees in CS/EE/CE and a strong background in DL, model training optimizations, and GPU architecture. Proficiency in C, Python, and CUDA is essential.

Qualifications

• MS or PhD in CS, EE or CE (or equivalent).
• Strong background in deep learning and model training optimizations.
• Deep background in computer architecture and GPU fundamentals.
• Proven experience analyzing and tuning application and system-level performance.
• Proficiency in C, Python, and CUDA.

Responsibilities

• Understand, analyze, profile, and optimize AI training workloads on advanced hardware and software platforms.
• Understand the big picture of training performance on GPUs, prioritizing and solving problems across state-of-the-art LLMs.
• Implement production-quality software in multiple layers of NVIDIA's deep learning platform stack.
• Implement key LLM training workloads in NVIDIA's processor and system simulators for future architecture studies.
• Build agentic tools to automate workload analysis, workload optimization, and other critical workflows.

Skills

Deep learning

Model training

Computer architecture

Performance modeling

C

Python

CUDA

Education

MS/PhD in Computer Science or Electrical/Computer Engineering

Job description

NVIDIA Corporation seeks an experienced High-Performance LLM Training Engineer to optimize its AI training workloads across frameworks like PyTorch and JAX for thousands of GPUs. You will profile, tune, and implement production-ready software across the deep learning stack and help shape future GPU architectures.

Required are advanced degrees in CS/EE/CE and a strong background in DL, model training optimizations, and GPU architecture. Proficiency in C, Python, and CUDA is essential.