Senior AI Factory Validation Architect

NVIDIA Corporation; Santa Clara (CA); Hybrid; USD 148,000 - 236,000; Full time

Job summary

NVIDIA Corporation in Santa Clara, CA seeks a Senior Solutions Architect to drive validation of AI factories from first rack power-on through customer handoff. You will run and debug AI/LLM workloads on Linux-based GPU clusters, collect evidence, and work with internal teams to ensure readiness for launch.

You’ll optimize NCCL usage, build observability, and automate benchmarks with Python and Shell. Strong collaboration and problem-solving skills are essential for scalable, high-performance AI

Qualifications

• Bachelor’s degree or equivalent in CS, Math, Engineering or related field.
• 5+ years of Linux-based HPC or distributed systems experience.
• Hands-on AI/ML workloads on multi-GPU clusters with NCCL.
• Strong knowledge of AllReduce/AllToAll patterns in ML training.

Responsibilities

• Set up and verify AI factory environments across multi-GPU/multi-node Linux clusters.
• Ensure configurations align with NCCL, collectives and distributed training frameworks.
• Own execution of AI/LLM benchmarks: setup, orchestration, result collection and analysis.
• Investigate and resolve training/benchmark issues
• Build observability for AI factories: metrics, logs, traces, dashboards
• Develop automation (Python, Shell) for benchmarks and regression checks
• Examine NCCL usage and communication patterns for ML workloads
• Recommend changes to configurations to improve throughput and scaling

Skills

Linux HPC

AI/ML workloads

NCCL

PyTorch

TensorFlow

Python

Shell/Bash

Benchmarking

Communication

Cross-functional teamwork

Education

Bachelor’s degree or equivalent in CS/Math/Engineering/Physics

Tools

NCCL

PyTorch

TensorFlow

Bash

Job description

NVIDIA Corporation in Santa Clara, CA seeks a Senior Solutions Architect to drive validation of AI factories from first rack power-on through customer handoff. You will run and debug AI/LLM workloads on Linux-based GPU clusters, collect evidence, and work with internal teams to ensure readiness for launch.

You’ll optimize NCCL usage, build observability, and automate benchmarks with Python and Shell. Strong collaboration and problem-solving skills are essential for scalable, high-performance AI