Principal Core Infrastructure Engineer - GPU Validation

Oracle Corporation; Nashville (TN); On-site; USD 140,000 - 180,000; Full time

Job summary

Oracle Corporation is seeking an HPC-focused engineer to develop automated tools and validation frameworks ensuring GPU racks are production-ready. You will build NCCL benchmarks, validate GPU infrastructure and NVLink tests, and create Slurm-based workflows for validation and diagnostics.

You will design, implement, and optimize distributed components with emphasis on scalability, resiliency, and operability, delivering features and loading tests while enabling automation and secure operations.

Qualifications

• Experience designing and optimizing distributed systems with focus on scalability and operability.
• Strong background in load and performance testing for HPC/GPU environments.
• Proficiency in automation/IaC and in implementing secure, compliant changes.

Responsibilities

• Build NCCL performance benchmarks and validate GPU racks for production.
• Develop GPU infrastructure and NVLink connectivity tests and Slurm-based validation workflows.
• Create automated FDT and cable diagnostics; design and optimize distributed components for resilience and operability.
• Deliver features, run load/performance tests, and review implementations for scalability.
• Construct fault-tolerant paths with replication and automatic failover; implement retries, circuit breakers, and timeouts.
• Write runbooks, participate in incident response and RCAs; apply telemetry, alarms, dashboards, and documentation practices.

Skills

Distributed systems

Scalability

Resiliency

Testing/validation

Automation

Security controls

Tools

NCCL benchmarks

Slurm

NVLink tests

Job description

Join the HPC team to develop automated tools and validation frameworks that ensure GPU racks are ready for production. The role includes building NCCL performance benchmarks, GPU infrastructure and NVLink connectivity tests, Slurm-based validation workflows, and automated FDT and cable diagnostics.

Designs, implements, and optimizes components in distributed systems with an emphasis on scalability, resiliency, and operability.

Designs, implements, and optimizes components in distributed systems with an emphasis on scalability, resiliency, and operability. Delivers features and load/performance tests; leverages data plane platforms and distributed state tools for high-volume retrieval, storage, and processing; and reviews peers’ implementations for scalability compliance. Builds fault-tolerant paths (redundancy, replication, automatic failover), applies recovery‑oriented principles, and implements retries, circuit breakers, and timeouts. Proactively detects and mitigates issues via tests, alarms, dashboards, and telemetry; authors runbooks and participates in incident response and RCAs. Implements standard replication and synchronization, develops automation/IaC for troubleshooting and maintenance, and applies advanced security controls (encryption, access, remediation) while ensuring change, compliance, and documentation standards are met.

Must live or be willing to relocate so you are able to regularly go into the Nashville, TN office.