Senior ML Performance Engineer, GPU Training & Inference

NVIDIA Corporation; Santa Clara (CA); On-site; USD 224,000 - 357,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA Corporation in Santa Clara, CA seeks a Software Engineer, ML Performance to optimize distributed training and inference workflows across GPUs. You will analyze bottlenecks, define metrics, and drive regression investigations with cross-team collaboration.

The role requires 12+ years of software engineering experience, strong Python and CUDA skills, and the ability to reason about signals across the stack. Equity and benefits are offered, with applications accepted through November 3, 2026.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, or related field, or equivalent experience.
• 12+ years of relevant software engineering experience, including hands-on optimization of GPU training or inference workloads in production.
• Demonstrated performance work across more than one layer of the stack with baseline, evidence, and validated outcomes.
• Experience diagnosing performance across many jobs, machines, or tenants, not just a single workload.
• Ability to discuss instrumentation signals and where to measure signals across frameworks, runtimes, and compilers.
• Strong programming and debugging skills with maintainable analysis code and repeatable procedures.
• Clear communication of findings, trade-offs and remaining uncertainty to workload owners.
• Proficiency with Python and CUDA.

Responsibilities

• Analyze the performance of distributed training and inference workloads across the stack to identify bottlenecks and propose solutions.
• Define performance metrics for a GPU fleet and determine essential signals to collect.
• Drive regression analysis by segmenting workloads across machines, versions, and configurations.
• Evaluate automated diagnoses, assess accuracy, and build labeled cases to improve reliability.
• Collaborate with build partners on real workloads and feed product gaps back into the roadmap.

Skills

Python

CUDA

Performance analysis

Debugging

Communication

Education

Bachelor's degree in CS/CE or equivalent

Tools

NCCL

PyTorch

JAX

Job description

NVIDIA Corporation in Santa Clara, CA seeks a Software Engineer, ML Performance to optimize distributed training and inference workflows across GPUs. You will analyze bottlenecks, define metrics, and drive regression investigations with cross-team collaboration.

The role requires 12+ years of software engineering experience, strong Python and CUDA skills, and the ability to reason about signals across the stack. Equity and benefits are offered, with applications accepted through November 3, 2026.