Principal LLM Training Performance Engineer | Equity

NVIDIA Corporation

Santa Clara (CA)

On-site

USD 272,000 - 431,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity
Benefits

Job summary

NVIDIA Corporation seeks a Principal Engineer to drive performance of large-scale AI training and post-training workloads across the full hardware and software stack. You will optimize transformer-based workloads on thousands of GPUs, shape GPU/software roadmaps, and mentor teams across CUDA, frameworks, and systems software.

You will operate across abstraction layers from application behavior to runtime internals, with responsibility for profiling, benchmarking, and performance modeling to

Qualifications

  • MS/PhD in Computer Science, Electrical Engineering, Computer Engineering, or related field; 12+ years of experience.
  • Principal-level technical impact in AI training systems, GPU performance optimization, distributed systems, or ML frameworks.
  • Hands-on experience optimizing performance of large-scale DL workloads (transformers/LLMs).
  • Strong understanding of GPU architecture from accelerators to datacenter-scale systems.
  • Experience with distributed training techniques and mixed precision training.
  • Proven ability to diagnose bottlenecks with profiling and benchmarking tools and drive measurable improvements.
  • Excellent communication and technical leadership across multiple teams.

Responsibilities

  • Lead end-to-end performance analysis and optimization of LLM pre-training and post-training workloads.
  • Identify bottlenecks across compute, memory, communication, scheduling, and parallelism strategies.
  • Develop production-quality software, tools, benchmarks, and analysis infra to improve training performance.
  • Build performance models and simulation methodologies to guide architecture decisions.
  • Serve as technical authority for AI training performance across teams.
  • Translate workload insights into hardware/software recommendations for performance gains.
  • Mentor engineers and establish best practices for large-scale AI performance analysis.

Skills

Deep learning performance
High-performance computing
Distributed training
Profiling & benchmarking
Technical leadership
Cross-team collaboration

Education

MS/PhD in Computer Science or Electrical/Computer Engineering

Tools

PyTorch
JAX
NeMo
NeMo RL
CUDA libraries
Profiling tools

Job description

NVIDIA Corporation seeks a Principal Engineer to drive performance of large-scale AI training and post-training workloads across the full hardware and software stack. You will optimize transformer-based workloads on thousands of GPUs, shape GPU/software roadmaps, and mentor teams across CUDA, frameworks, and systems software.

You will operate across abstraction layers from application behavior to runtime internals, with responsibility for profiling, benchmarking, and performance modeling to

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior LLM Training Performance Engineer
Senior LLM Training Performance Engineer

NVIDIA Corporation • California

On-site
USD 184,000 - 357,000
Equity
Benefits package
Inclusive culture
Principal High-Performance LLM Training Engineer
Principal High-Performance LLM Training Engineer

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 272,000 - 431,250
Equity
Benefits
Lead High-Performance LLM Training Architect (Equity)
Lead High-Performance LLM Training Architect (Equity)

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 272,000 - 431,250
Equity
Benefits
LLM Training Performance Engineer (GPU Optimization)
LLM Training Performance Engineer (GPU Optimization)

NVIDIA Corporation • Town of Montana (WI)

On-site
USD 124,000 - 242,000
Principal High-Performance LLM Training Engineer
Principal High-Performance LLM Training Engineer

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 272,000 - 431,000
Equity
Benefits
Senior High-Performance LLM Training Engineer (Hybrid, Equity)
Senior High-Performance LLM Training Engineer (Hybrid, Equity)

NVIDIA Corporation • Santa Clara (CA)

Hybrid
USD 184,000 - 357,000
Equity
Benefits package
Hybrid work option
Senior High-Performance LLM Training Engineer
Senior High-Performance LLM Training Engineer

NVIDIA Corporation • Santa Clara (CA)

Hybrid
USD 184,000 - 356,500
Equity
Benefits
Senior High-Performance LLM Training Engineer
Senior High-Performance LLM Training Engineer

NVIDIA Corporation • Santa Clara (CA)

Hybrid
USD 184,000 - 357,000
Equity
Benefits package
Hybrid work option
Senior High-Performance LLM Training Engineer
Senior High-Performance LLM Training Engineer

NVIDIA Corporation • California

On-site
USD 184,000 - 357,000
Equity
Benefits package
Inclusive culture
High-Performance LLM Training Engineer - New College Grad 2027
High-Performance LLM Training Engineer - New College Grad 2027

NVIDIA Corporation • Town of Montana (WI)

On-site
USD 124,000 - 242,000