Site Reliability Engineer - Datacenter

SpaceXAI; Memphis (TN); On-site; USD 110,000 - 160,000; Full time

Job summary

SpaceXAI is seeking a Site Reliability Engineer focused on Hardware to drive reliability across our Memphis datacenter. You will own firmware analysis, hardware spec review, and vendor relations while proactively identifying issues before they impact operations.

You will collaborate with Datacenter Operations Technicians, develop monitoring tools, handle RMAs, and document RCAs and AFRs to build a durable knowledge base for the team.

Qualifications

• Bachelor's degree in a related field or equivalent experience.
• 2+ years in hardware reliability engineering in HPC or datacenter environments.
• Proven firmware analysis and hardware spec review experience.
• Experience with RMA processes and vendor negotiations.
• Ability to diagnose grey or intermittent hardware failures with data.
• Familiarity with datacenter components (servers, GPUs, networking).
• Scripting in Python or Bash for automation; knowledge of at least one systems language.
• Strong problem-solving and data-driven reliability mindset.
• Collaborative, cross-functional teamwork.

Responsibilities

• Analyze firmware packages and hardware specs for releases, run security scanning and CVE analysis.
• Investigate and diagnose hardware failures, proving defects via testing and data.
• Manage vendor relationships and initiate RMAs.
• Collaborate with Datacenter Operations Technicians to troubleshoot hardware.
• Develop monitoring tools and scripts to detect anomalies and reduce downtime.
• Document failure modes, RCAs, AFRs, and RMA outcomes in knowledge base.
• Participate in on-call rotations and incident response in Memphis datacenter.

Skills

Firmware analysis

Hardware specifications review

Release validation

RMA processes

Failure analysis

Scripting: Python

Shell scripting: Bash

Programming: C/C++/Java/Rust

Data-driven reliability

Cross-functional collaboration

Education

Bachelor's degree in Systems Engineering, Electrical Engineering, Computer Science

Tools

Logic analyzers

Diagnostic software

Vendor management tools

Job description

SpaceXAI’s mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. Our team is small, highly motivated, and focused on engineering excellence. This organization is for individuals who appreciate challenging themselves and thrive on curiosity. We operate with a flat organizational structure. All employees are expected to be hands-on and to contribute directly to the company’s mission. Leadership is given to those who show initiative and consistently deliver excellence. Work ethic and strong prioritization skills are important. All employees are expected to have strong communication skills. They should be able to concisely and accurately share knowledge with their teammates.

ABOUT THE ROLE:

As a Site Reliability Engineer focused on Hardware, you will serve as an expert focused on firmware, hardware specifications, vendor relations, and failure analysis. You will proactively identify and resolve hardware issues, manage RMA processes, and stay ahead of emerging hardware technologies to support SpaceXAI's datacenter operations. This role demands deep technical expertise in hardware diagnostics, and forward-looking hardware evaluation.

RESPONSIBILITIES:

• Analyze firmware packages and hardware specifications for upcoming releases for compatibility, performance, and reliability in SpaceXAI's datacenter environment. Run security scanning and CVE / vulnerability analysis on firmware and related components. Flag safety issues (electrical, thermal, power-protection, fail-safe behavior) before the package hits the floor.
• Investigate and diagnose hardware failures, including "grey failures" (ambiguous or intermittent issues), proving them as true hardware defects through rigorous testing and data analysis.
• Manage vendor relationships, including initiating RMA (Return Merchandise Authorization) claims, negotiating beyond standard processes when necessary, and holding vendors accountable for resolutions.
• Collaborate with Datacenter Operations Technicians to troubleshoot, repair, and optimize hardware systems in real-time.
• Develop and implement monitoring tools, scripts, and processes to detect hardware anomalies early and minimize downtime.
• Document failure modes, RCAs, AFR / reliability models, RMA outcomes, and hardware evaluations into a team knowledge base.
• Participate in on-call rotations and incident response for hardware-related issues in the Memphis datacenter

BASIC QUALIFICATIONS:

• Bachelor's degree in Systems Engineering, Electrical Engineering, Computer Science, or a related field (or equivalent experience).
• 2+ years of experience in hardware reliability engineering, preferably in high-performance computing or datacenter environments.
• Proven expertise in firmware analysis, hardware specifications review, and release validation.
• Strong experience with RMA processes, including filing claims, vendor negotiations, and pushing for resolutions outside standard protocols.
• Demonstrated ability to diagnose and prove complex hardware failures, including grey or intermittent issues, using tools, logic analyzers, or diagnostic software.
• Familiarity with datacenter hardware components (e.g., servers, GPUs, networking equipment) and emerging technologies.
• Proficiency in scripting (Python, Bash) for automation and analysis, plus general experience in at least one systems language (C, C++, Java, Rust, or similar). Not required to be expert in all of them.
• Excellent problem-solving skills with a data-driven approach to reliability engineering.
• Ability to work collaboratively with cross-functional teams, including operations technicians.

PREFERRED SKILLS AND EXPERIENCE:

• Experience in AI/ML infrastructure or supercomputing environments.
• Knowledge of vendor ecosystems (e.g., NVIDIA, Dell, HP, Supermicro) and supply chain management.
• Certifications in hardware engineering or reliability (e.g., CRE, CompTIA Server+).
• Prior work in a fast-paced startup or tech company like SpaceXAI.

SpaceXAI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.