Hardware Reliability Engineer

Socrates Systems Inc

United States

Remote

USD 180,000 - 240,000

Full time

11 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Competitive compensation
Equity
Medical, dental, vision
Flexible time off
Company breaks

Job summary

Socrates is seeking a Senior GPU Infra Engineer to own the zero-to-one build-out of Socrates GRID, designing compute, network, storage, and management architecture for large-scale deployment. You will drive SRE practices and ensure performance, availability, and cost targets are met.

Lead provisioning automation, capacity planning, and multi-site expansion, coordinating with platform engineers and OEMs. Hands-on GPU hardware expertise is essential for production-scale clusters.

Qualifications

  • Must have built and operated production GPU infrastructure.
  • Deep understanding of GPU memory, interconnects, CPU topology, PCIe lanes, and power/thermals.
  • Ability to trace bottlenecks in the physical system.
  • Experience with high-speed cluster networking (InfiniBand or RoCE).
  • Experience shipping production software and automation in Python or Go.

Responsibilities

  • Design compute, network, storage, and management architecture for Socrates GRID.
  • Bring first capacity online and make expansions repeatable.
  • Own SRE with measurable performance, availability, recovery, and cost.
  • Automate provisioning, configuration, health checks, upgrades, failure isolation.
  • Define service-level objectives and incident reviews.

Skills

GPU infra
Server selection
GPU memory
GPU diagnostics
High-speed networking
Python
Go
SRE
Inference ops
Vendor coordination

Tools

CI/CD pipelines

Job description

About us

Socrates is an AI-native platform for a new generation of business software. We bring models, data, agents, and applications together so companies can replace disconnected systems with software built around how they work. The underlying infrastructure is built in, giving businesses one foundation to manage their records, run their workflows, and put AI to work.

Our mission is to give businesses greater control over their technology and make it easier for people to create and use software with AI. We’ve raised a $23 million seed round to help build that future.

The Role

Own the zero-to-one build-out of Socrates GRID and the path to large-scale deployment. Design the compute, network, storage, and management architecture, bring the first capacity online, and make each expansion repeatable. Integrate the fleet with Socrates deployment, identity, observability, and inference workflows. Own site reliability engineering (SRE), with measurable performance, availability, recovery, and operating cost.

Initial priorities:

  • Define and deliver the initial infrastructure: server specifications, GPU and network topology, storage, rack layouts, power and cooling requirements, and acceptance tests.

  • Integrate GPU capacity with Socrates deployment, identity, observability, and inference workflows. Ship the platform changes and developer tools needed to use and operate it.

  • Run representative models and measure serving throughput, latency, availability, power, utilization, and cost per million tokens.

  • Automate provisioning, configuration, health checks, upgrades, failure isolation, and recovery.

  • Establish SRE practices, including service-level objectives, actionable alerts, incident reviews, and tested recovery procedures.

  • Build the operating model for multi-rack and multi-site expansion, including capacity planning, vendor acceptance, spares, repairs, maintenance, and failure containment.

Must-haves
  • Has personally built and operated production GPU infrastructure. Can explain server selection, rack integration, commissioning, and the changes required as a deployment grew.

  • Understands the hardware in depth: GPU memory and interconnects, CPU and NUMA topology, PCIe lanes, NIC placement, storage paths, firmware, power, and thermals. Can trace a workload bottleneck through the physical system.

  • Diagnoses GPU and fabric failures using hardware evidence, including NVIDIA Xid and ECC reports, DCGM diagnostics, PCIe errors, NVLink health, NIC and switch counters, and BMC logs. Can separate a component failure from a driver, topology, or configuration problem.

  • Has hands-on experience with high-speed cluster networking, such as InfiniBand or RoCE. Can validate bandwidth, latency, congestion, and collective performance rather than relying on link-up status.

  • Ships production software and maintainable automation in Python, Go, or a comparable language. Can contribute platform features and developer tools alongside bare-metal provisioning, configuration, firmware and driver compatibility, and out-of-band hardware access.

  • Delivers in short iterations at the team's pace. Uses agents effectively, keeps changes reviewable, and follows work through CI, deployment, and operating checks. Contributes to shared engineering priorities beyond the hardware backlog.

  • Applies SRE practices to GPU capacity: meaningful service objectives, monitoring, incident response, tested recovery, and maintenance that limits customer disruption.

  • Understands inference operations, including model memory requirements, batching, concurrency, and the tradeoffs between throughput, latency, and utilization. Can measure useful work rather than treating GPU activity as business value.

  • Can coordinate OEMs, colocation operators, and platform engineers through installation, acceptance, repairs, and expansion. Owns technical decisions with incomplete direction and records evidence for others to review.

Why join us

Join Socrates to help shape how businesses build, use, and own software and AI. You’ll tackle hard problems in making AI reliable, affordable, and secure enough to run the systems businesses depend on. Your decisions will help shape our technology, our products, and everything built on the platform. We value people who take responsibility, move with urgency, and make the work around them better. This is an opportunity to make a lasting contribution to the product and the company behind it.

Benefits & Perks

We’ve got you covered with competitive compensation, equity, medical, dental, and vision benefits, plus flexible time off coordinated with your team. Alongside federal holidays, you’ll enjoy company-wide summer and winter breaks—dedicated time for everyone to unplug, recharge, and make the most of life outside work.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Finance and Operations Manager
Finance and Operations Manager

Socrates Systems Inc • United States

Remote
USD 120,000 - 190,000
Equity
Medical, dental, and vision benefits
Flexible time off
+1
Lead GPU Infrastructure Reliability Engineer
Lead GPU Infrastructure Reliability Engineer

Socrates • United States

Remote
USD 180,000 - 240,000
Competitive compensation
Equity
Medical, dental, vision
+2
Hyperbolic Labs - Senior Site Reliability Engineer
Hyperbolic Labs - Senior Site Reliability Engineer

deCircle • San Francisco (CA)

On-site
USD 120,000 - 150,000
Staff Technical Recruiter
Staff Technical Recruiter

Socrates • United States

Remote
USD 140,000 - 220,000
Competitive compensation
Equity
Medical, dental, vision benefits
+3
Staff Technical Recruiter
Staff Technical Recruiter

Socrates Systems Inc • United States

Remote
USD 150,000 - 210,000
Equity
Medical benefits
Flexible time off
Senior Technical Recruiter — Own Tech Hiring & Strategy
Senior Technical Recruiter — Own Tech Hiring & Strategy

Socrates • United States

Remote
USD 140,000 - 220,000
Competitive compensation
Equity
Medical, dental, vision benefits
+3
Site Reliability Engineer
Site Reliability Engineer

Nscale • San Francisco (CA)

On-site
USD 130,000 - 200,000
Equity
Career progression plan
Flexible schedule
Senior Site Reliability Engineer
Senior Site Reliability Engineer

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 168,000 - 270,000
Equity
Benefits package
Procurement & Supply Chain Lead, Hardware
Procurement & Supply Chain Lead, Hardware

Socrates Systems Inc • United States

Remote
USD 170,000 - 250,000
Equity
Medical, dental, and vision benefits
401(k) or equivalent
+2
Reliability Engineer, Supercomputing
Reliability Engineer, Supercomputing

Bison • San Francisco (CA)

On-site
USD 350,000 - 475,000
Health, dental, and vision benefits
Unlimited PTO
Paid parental leave
+1