Head of Infrastructure (Remote)

General Compute

San Francisco (CA)

Remote

USD 180,000 - 250,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

General Compute seeks an experienced infrastructure/Platform Engineer to own the gateway and load balancer for our ASIC fleet and to oversee the observability stack across a growing heterogeneous inference infrastructure. You will drive capacity planning against real, distributed traffic and manage the operational relationship with our ASIC partner, bringing in-house expertise as the system scales.

Ideal candidates have 7+ years in infrastructure, SRE, or platform engineering within ML, HPC, or

Qualifications

  • 7+ years in infrastructure, SRE, or platform engineering, with at least some time at a serious inference, ML, or HPC shop.
  • Track record of building observability practices that catch problems, not just dashboards.
  • Background with model-serving stacks (vLLM, TGI, TensorRT-LLM, SGLang). Network fabric experience at data-center scale (RoCE, InfiniBand).

Responsibilities

  • Own the gateway and load balancer that fronts the ASIC fleet.
  • Run capacity planning against a real, distributed traffic mix across open-weight models.
  • Own the operational side of the ASIC partnership and be the technical face in partner conversations.

Skills

Infrastructure engineering
SRE experience
Platform engineering

Job description

Job description

Inference is fragmenting: purpose-built silicon from SambaNova, Cerebras, Positron, d-Matrix, and others already beats GPUs on decode, and we productionize that hardware — we buy the racks, find the data center space, and run it for our customers. Our customers are frontier labs, fast-growing AI application companies, and asset-light clouds. We closed a $15M seed round in May 2026, and have since closed a $400M debt facility — $100M funded upfront by Upper90, with the balance available for drawdown — collateralized by our inference chips. You’ll own the infrastructure layer of our inference cloud end-to-end. Today that means the control plane, the gateway in front of our ASIC fleet, and the observability stack that tells us where every millisecond goes. Over the next 6-8 months, it will grow into a heterogeneous fleet: At the moment, it’s built on configuration provided by our ASIC partner; Own the gateway and load balancer that fronts the fleet. Run capacity planning against a real, distributed traffic mix across the open-weight models we serve. Own the operational side of the ASIC partnership. Most weird production issues route through their engineering team until we build that expertise in-house, and you’ll be our technical face in those conversations. Bring up the pre-fill side of our disaggregated architecture on a second hardware platform as it comes online.

  • 7+ years in infrastructure, SRE, or platform engineering, with at least some of it at a serious inference, ML, or HPC shop.
  • Track record of building observability practices that actually catch problems, not just generate dashboards.
  • Background with model-serving stacks (vLLM, TGI, TensorRT-LLM, SGLang). Network fabric experience at data-center scale (RoCE, InfiniBand).
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Head of Infrastructure
Head of Infrastructure

General Compute • San Francisco (CA)

On-site
USD 190,000 - 280,000
Head of Inference Infrastructure
Head of Inference Infrastructure

General Compute • San Francisco (CA)

On-site
USD 190,000 - 280,000
Head of Infrastructure
Head of Infrastructure

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Founding Platform Engineer
Founding Platform Engineer

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Platform Engineer
Platform Engineer

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Inference Engineer
Inference Engineer

Black.ai • San Francisco (CA)

On-site
USD 190,000 - 240,000
Founding Inference Engineer
Founding Inference Engineer

General Compute • San Francisco (CA)

On-site
USD 190,000 - 230,000
Head of Inference Cloud Infrastructure
Head of Inference Cloud Infrastructure

General Compute • San Francisco (CA)

Remote
USD 180,000 - 250,000
Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure)
Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure)

Perplexity • United States

Remote
USD 180,000 - 320,000
Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure)
Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure)

Perplexity • United States

On-site
USD 180,000 - 240,000