Principal E2E Performance Engineer for CSPs

NVIDIA Corporation; Santa Clara, Northern (CA, KY); Hybrid; USD 272,000 - 431,000; Full time

Job summary

NVIDIA is seeking a Principal Software Engineer to lead end-to-end performance initiatives for CSP engagements. You will coordinate with CSPs and NVIDIA teams to understand workloads, profile performance, and drive optimizations across CUDA, NCCL, drivers, and firmware.

The role emphasizes deep experience in GPU/HPC performance, statistical analysis, and scalable benchmarking. You will influence tooling, documentation, and validation strategies to meet customer targets and advance platform

Qualifications

• 15+ years of systems performance engineering experience.
• Proficiency with GPU workload profiling tools (Nsight Systems, Nsight Compute, DCGM metrics).
• Strong understanding of distributed training performance dynamics and memory bandwidth utilization.
• Statistical methods for performance analysis (regression, confidence intervals, A/B comparison).
• Understanding of how full software stack impacts performance (driver overhead, memory allocation, scheduling).
• Strong data analysis and visualization skills (Python, pandas, dashboards).

Responsibilities

• Drive performance characterization work streams with CSP engineering teams, ensuring they understand platform performance expectations and tuning options.
• Gather CSP feedback and champion optimization priorities back into NVIDIA's CUDA, NCCL, driver, and firmware teams.
• Ensure open-source tools (STREAM, GPU Burn, GPU BLAST) are updated and validated for latest NVIDIA rack-scale systems.
• Collaborate with CSPs to keep performance tooling aligned with latest GPU capabilities and configurations.
• Define test strategies and tooling requirements for performance validation for internal certification and customer acceptance.

Skills

Performance profiling

Python

Pandas

Data analysis

Distributed training

Statistical methods

Education

BS/MS in CS/CE or related field

Tools

Nsight Systems

Nsight Compute

DCGM metrics

Profiling tooling

Job description

NVIDIA is seeking a Principal Software Engineer to lead end-to-end performance initiatives for CSP engagements. You will coordinate with CSPs and NVIDIA teams to understand workloads, profile performance, and drive optimizations across CUDA, NCCL, drivers, and firmware.

The role emphasizes deep experience in GPU/HPC performance, statistical analysis, and scalable benchmarking. You will influence tooling, documentation, and validation strategies to meet customer targets and advance platform