Senior NCX Engineer: Day 2 Ops & Observability

NVIDIA; Santa Clara (CA); On-site; USD 184,000 - 357,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA is hiring an NCX Senior Engineer to join the DSX team, focusing on Day 2 operations for NVIDIA Cloud Partners. You’ll build observability, automation, and production practices across GPU clusters and cloud infrastructure to improve reliability in training and inference workloads.

You will collaborate with partner teams, translate reference architectures into runbooks, and drive scalable lifecycle management for large multi-node environments.

Qualifications

• BS, MS, or Ph.D. in Computer Science, Computer/Electrical Engineering, or related field, or equivalent experience.
• 8+ years in infrastructure engineering, SRE, DevOps, or similar large-scale production roles.
• Strong Linux-based distributed systems and cloud infrastructure experience.
• Deep understanding of Kubernetes, containers, cluster scheduling, and large multi-node environments.
• Experience with production observability: metrics, logging, dashboards, SLAs, and alerts.

Responsibilities

• Lead Day 2 operational readiness with NVIDIA Cloud Partners and define repeatable processes.
• Build continuous validation across GPU/CPU/storage/network health for large AI clusters.
• Establish telemetry, monitoring dashboards, and operational signals across stacks.
• Develop automated remediation workflows to minimize disruption to workloads.
• Refine fleet lifecycle administration: drivers, firmware, OS patching, drift detection.
• Operationalize NVIDIA reference architectures into runbooks and practices.
• Define health signals, SLOs, and acceptance criteria for reliability.

Skills

Kubernetes

Linux

Python

Go

Shell scripting

Observability

Cloud infrastructure

SRE

DevOps

Education

BS/MS/PhD in Computer Science or related field

Tools

Prometheus

Grafana

OpenTelemetry

Alertmanager

Kubernetes

Job description

NVIDIA is hiring an NCX Senior Engineer to join the DSX team, focusing on Day 2 operations for NVIDIA Cloud Partners. You’ll build observability, automation, and production practices across GPU clusters and cloud infrastructure to improve reliability in training and inference workloads.

You will collaborate with partner teams, translate reference architectures into runbooks, and drive scalable lifecycle management for large multi-node environments.