Senior Resiliency Architect, Hardware Diagnostics and Resiliency Modeling

2100 NVIDIA USA

Santa Clara (CA)

Sur place

USD 184 000 - 288 000

Plein temps

14 jours+

Une candidature complète en une minute

Un CV et une lettre de motivation personnalisés, prêts à envoyer.

Passez les filtres ATS

Avantages offerts par ce poste

Equity
Benefits

Résumé du poste

NVIDIA Corporation seeks a Senior Resiliency Architect to advance hardware diagnostics and resiliency models for Datacenter and Edge products, spanning GPUs, SOCs, and AI workloads. You will drive robust fault attribution and improve test time, coverage, and telemetry accuracy while collaborating with cross-disciplinary teams.

You will leverage Python/CUDA tooling to analyze field data, build dashboards, and develop automation for scalable diagnostic workflows across racks and datacenters.

Qualifications

  • Master's degree in Computer Engineering, Electrical Engineering, Computer Science or closely related degree or equivalent experience.
  • Minimum 5+ years of relevant experience in resiliency, diagnostics, or hardware reliability.
  • Strong knowledge of GPU architecture, hardware diagnostics and reliability models (FIT) and AVF/Availability concepts.

Responsabilités

  • Develop and maintain diagnostics software to stress test NVIDIA GPUs and SOCs, identifying defects including silent data corruption.
  • Run tests in large-scale deployments of Datacenter GPUs and Safety SOCs across GPUs, CPUs, and Networking SOCs.
  • Analyze coverage gaps in the diagnostic suite and improve test coverage, repeatability, and time efficiency.
  • Model and analyze FIT-based reliability and availability metrics across GPU to Datacenter levels.
  • Design and maintain automation scripts to kick off and monitor diagnostic jobs across compute clusters.
  • Develop dashboards that integrate diverse data sources and enable analytics and secure multi-user access.
  • Collaborate with engineering to ensure telemetry correctly categorizes failures.

Connaissances

GPU Architecture
Hardware Diagnostics
Reliability models (FIT)
AVF estimation
Availability concepts
ECC/parity/CRC
Python scripting
SQL
C/C++
CUDA
AI tooling
Interpersonal skills
Collaboration

Formation

Master's degree (computing/engineering)

Outils

CUDA toolkit
Python

Description du poste

We are now seeking a Senior Resiliency Architect to support the development of hardware diagnostics and resiliency models for Datacenter and Edge Computing products with applications spanning Agentic and Physical AI. In this role, you will be a key member of a team of innovators, challenging the status quo and pushing beyond boundaries. You will have the opportunity to impact the industry's leading GPUs and SoCs powering product lines ranging from the rapidly growing field of artificial intelligence to self-driving cars and robots.

What you'll be doing:
  • Develop and maintain diagnostics software suite to efficiently stress test NVIDIA GPUs and SOCs to identify hardware defects, including defects that cause silent data corruption.
  • These tests will run in large-scale deployments of Datacenter GPUs and Safety SOCs in package/board/rack configurations spanning GPUs, CPUs, and Networking SOCs.
  • Analyze coverage gaps in NVIDIA diagnostic suite flagged by silicon failures on customer workloads or test suites.
  • Enhance diagnostics to improve coverage, repeatability of failures detected, and optimize test time.
  • Model and analyze RAS metrics like Failures in Time (FIT) for permanent and transient errors, and Availability from GPU to Rack to Datacenter.
  • Use models to identify gaps and drive RAS improvements through new hardware and software features.
  • Analyze field data on hardware interruptions and permanent failures, enhance RAS models to better correlate to field data.
  • Ensure optimal fault attribution, containment, and recovery for hardware errors.
  • Design, build, and maintain automation scripts and tooling to systematically kick off, monitor, and manage diagnostic jobs across multiple independent compute clusters.
  • Develop and maintain dashboards that integrate diverse data sources and formats, deliver insights through analytics and visualization, and ensure reliable hosting, secure access, and multi-user performance.
  • Collaborate with engineering teams to ensure system telemetries accurately categorize cluster failures into hardware faults, test-script bugs, or environment configuration issues.
  • Ensure availability models and hardware diagnostics evolve to reflect AI workload changes
What we need to see:
  • Master's degree in Computer Engineering, Electrical Engineering, Computer Science or closely related degree or equivalent experience.
  • At least 5+ years of relevant experience.
  • Knowledge of GPU Architecture and programming, hardware diagnostics Good understanding of Reliability (FIT) models, AVF estimation, Availability concepts, ECC/parity/CRC strategies.
  • Proficiency in data mining, automation, usage of AI tools and scripting with Python, SQLor similar.
  • Proficiency in C/C++, CUDA programming, and using AI tools for coding and debug.
  • Strong debugging and analytical skills.
  • Excellent interpersonal skills and ability to collaborate with on-site and remote teams.
  • Working knowledge of hardware diagnostics, and container-based job execution.
Ways to stand out from the crowd:
  • Strong statistical judgment - particularly around exposure-adjusted rates, sample size, and distinguishing signal from artefact.
  • Strong understanding of resiliency and reliability trade-offs in AI data centers, including failure modes, mitigation and recovery strategies, and impact on large-scale training/inference workloads
  • Experience in developing embedded software or visualization and data analytics tools.

Your base salary will be determined based on your location, experience, and other objective job-related factors.

The expected base salary range for this position is 184,000 USD - 287,500 USD.

You will also be eligible for equity and benefits.

Applications for this job will be accepted at least until November 7, 2026.

This posting is for an existing vacancy.

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer.

As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

NVIDIA pioneered accelerated computing.

Today, our AI infrastructure powers global intelligence, transforming every industry.

Learn more about NVIDIA.

Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Senior Resiliency Architect,HardwareDiagnosticsand Resiliency Modeling
Senior Resiliency Architect,HardwareDiagnosticsand Resiliency Modeling

Nvidia Corporation in • Santa Clara (CA)

Sur place
USD 184 000 - 288 000
Senior Resiliency Architect, Hardware Diagnostics and Resiliency Modeling
Senior Resiliency Architect, Hardware Diagnostics and Resiliency Modeling

NVIDIA Corporation • États-Unis

Hybride
USD 184 000 - 288 000
Equity
Benefits
Senior Resiliency Architect,HardwareDiagnosticsand Resiliency Modeling
Senior Resiliency Architect,HardwareDiagnosticsand Resiliency Modeling

NVIDIA Corporation • Santa Clara (CA)

Sur place
USD 184 000 - 288 000
Equity
Benefits
Senior Software Engineer, Resilience Engineering - DGX Cloud
Senior Software Engineer, Resilience Engineering - DGX Cloud

NVIDIA Corporation • Santa Clara (CA)

Sur place
USD 184 000 - 357 000
Equity
Competitive benefits
System Software Engineer – Data Center GPU Compute Diagnostics
System Software Engineer – Data Center GPU Compute Diagnostics

NVIDIA Corporation • North Carolina

Sur place
USD 124 000 - 196 000
Senior System Software Engineer, Enterprise MODS
Senior System Software Engineer, Enterprise MODS

NVIDIA Corporation • Santa Clara (CA)

Sur place
USD 184 000 - 357 000
Equity
Benefits
System Software Engineer – Data Center GPU Compute Diagnostics
System Software Engineer – Data Center GPU Compute Diagnostics

NVIDIA Corporation • Durham (NC)

Sur place
USD 124 000 - 196 000
Equity
Benefits
Senior System Software Engineer, DGX Cloud AI Infrastructure
Senior System Software Engineer, DGX Cloud AI Infrastructure

2100 NVIDIA USA • Santa Clara (CA)

Sur place
USD 224 000 - 431 000
Senior Manager, Failure Analysis
Senior Manager, Failure Analysis

NVIDIA Corporation • Santa Clara (CA)

Sur place
USD 232 000 - 368 000
Senior System Software Engineer, DGX Cloud AI Infrastructure
Senior System Software Engineer, DGX Cloud AI Infrastructure

Nvidia Corporation in • Santa Clara (CA)

Sur place
USD 224 000 - 357 000
Equity
Benefits