Senior HPC Infra Engineer: GPU, Slurm & NVLink (Nashville)

Devopsroles; Nashville (TN); In loco; USD 115.000 - 235.000; Tempo pieno

Vantaggi offerti da questo lavoro

Medical, dental, and vision insurance

Short and long-term disability

Life insurance and AD&D

401(k) with company match

Paid time off and vacation

Descrizione del lavoro

Oracle is seeking an experienced HPC validation engineer in Nashville to design and implement automated tools and validation frameworks for GPU racks and HPC infrastructure. The role focuses on building NCCL benchmarks, NVLink connectivity tests, Slurm workflows, and automated diagnostics, while ensuring change control and security practices.

You will develop scalable validation software, dashboards, and runbooks, and contribute to incident response and RCA investigations.

Competenze

• Experience designing validation frameworks for HPC and GPU racks.
• Proficiency with Bash, Python, and IaC tooling.
• Ability to implement monitoring, telemetry, and runbooks.

Mansioni

• Design and develop scalable validation frameworks for GPU racks and HPC infrastructure.
• Build automated benchmarks and diagnostics for NCCL performance, GPU health, NVLink connectivity, and high-speed networks.
• Develop Slurm-based workflows for distributed validation and performance testing.
• Implement retries, timeouts, and recovery mechanisms to improve resilience.
• Develop functional, load, and fault-injection tests for reliability and production readiness.
• Build dashboards, telemetry, and alerts for infrastructure health and validation results.
• Develop Linux diagnostics tools, FDT, and cable validation; write incident runbooks.
• Automate patching, updates, and rollback workflows with IaC; maintain docs.

Conoscenze

Distributed systems

Bash

Python

Infrastructure as Code

Telemetry

Incident response

Strumenti

Slurm

NCCL

InfiniBand

NVLink

FDT

cable diagnostics

Descrizione del lavoro

Oracle is seeking an experienced HPC validation engineer in Nashville to design and implement automated tools and validation frameworks for GPU racks and HPC infrastructure. The role focuses on building NCCL benchmarks, NVLink connectivity tests, Slurm workflows, and automated diagnostics, while ensuring change control and security practices.

You will develop scalable validation software, dashboards, and runbooks, and contribute to incident response and RCA investigations.