Hardware Failure Analysis Engineer

Precision Labs; Southaven (MS); Hybrid; USD 90,000 - 140,000; Full time

Job summary

SpaceXSI seeks a hardware reliability engineer to diagnose fleet-wide failures, drive fixes with vendors, and improve RMA and spare strategies across data-center hardware. Candidates should have strong firmware analysis skills and experience with GPU, NIC, memory, and power systems.

The role requires collaboration with cross-functional teams, scripting for automation, and a data-driven approach to reliability in high-performance computing environments.

Qualifications

• Bachelor's degree in Systems Engineering, Electrical Engineering, Computer Science, or related field (or equivalent experience).
• 2+ years of hardware reliability engineering experience in high-performance computing or data centers.
• Proven firmware analysis, hardware specs review, and release validation experience.

Responsibilities

• Own named failure classes across GPU trays/baseboards, NIC/DPU, motherboard/PCIe switch, memory, power, thermal, cables/connectors, and rack-scale patterns.
• Run recurrence analysis and fleet-wide defect clustering; detect systemic patterns before they become fleet-scale loss.
• Build vendor escalation packages with evidence quality to drive action; collaborate with OEM/ODM manufacturers on firmware issues and manufacturing escapes.
• Feed findings into RMA policy, spare strategy, and stop-rules.
• Manage FA intake queue on rotation; keep queue age within SLA.

Skills

Python scripting

C/C++

Firmware analysis

RMA processes

Diagnostics tools

Data center hardware

Root cause analysis

Vendor negotiations

Education

Bachelor's degree in Systems Engineering / Electrical Engineering / CS

Tools

Logic analyzers

Diagnostic software

Job description

SpaceX was founded under the belief that a future where humanity is out exploring the stars is fundamentally more exciting than one where we are not. Today, SpaceX is actively developing the technologies to make this possible, with the ultimate goal of making life multiplanetary and expanding consciousness to the stars.
SpaceXSI is building Super Intelligence to help humanity understand the universe. Our teams are building the full Super Intelligence stack: from our Colossus supercomputers, to Grok models, to our suite of applications, including Grok Bot, Grok Build, Grok Imagine, Grok Voice, and X. We're creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next.

ABOUT THE ROLE:

Determine true root cause of fleet hardware failures — component and system level — and drive fixes through vendors to the manufacturer. Turn "swap it again" into "vendor redesign / firmware fix / manufacturing escape found."

RESPONSIBILITIES:

• Own named failure classes across GPU trays/baseboards, NIC/DPU, motherboard/PCIe switch, memory, power, thermal, cables/connectors, and rack-scale patterns.
• Run recurrence analysis and fleet-wide defect clustering; detect systemic patterns before they become fleet-scale loss.
• Build vendor technical escalation packages with evidence quality that forces action; partner with OEM/ODM/component manufacturers on firmware, bring-up, and manufacturing escapes through committed CAPA.
• Feed findings into RMA policy, spare strategy, and "do not reseat forever" stop-rules.
• Work the FA intake queue on rotation; keep queue age within SLA

BASIC QUALIFICATIONS:

• Bachelor's degree in Systems Engineering, Electrical Engineering, Computer Science, or a related field (or equivalent experience).
• 2+ years of experience in hardware reliability engineering, preferably in high-performance computing or data center environments.
• Proven expertise in firmware analysis, hardware specifications review, and release validation.
• Strong experience with RMA processes, including filing claims, vendor negotiations, and pushing for resolutions outside standard protocols.
• Demonstrated ability to diagnose and prove complex hardware failures, including grey or intermittent issues, using tools, logic analyzers, or diagnostic software.
• Familiarity with data center hardware components (e.g., servers, GPUs, networking equipment) and emerging technologies.
• Proficiency in scripting (Python, Bash) for automation and analysis, plus general experience in at least one systems language (C, C++, Java, Rust, or similar). Not required to be expert in all of them.
• Excellent problem-solving skills with a data-driven approach to reliability engineering.
• Ability to work collaboratively with cross-functional teams, including operations technicians.

PREFERRED SKILLS AND EXPERIENCE:

• Experience in AI/ML infrastructure or supercomputing environments.
• Knowledge of vendor ecosystems (e.g., NVIDIA, Dell, HP, Supermicro) and supply chain management.
• Certifications in hardware engineering or reliability (e.g., CRE, CompTIA Server+).
• Prior work in a fast-paced startup or tech company like SpaceXSI.

SpaceXSI is an equal opportunity employer. For details on data processing, view our Recruitment Privacy Notice.