Head of Inference Cloud Infrastructure

General Compute

San Francisco (CA)

Remote

USD 180,000 - 250,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

General Compute seeks an experienced infrastructure/Platform Engineer to own the gateway and load balancer for our ASIC fleet and to oversee the observability stack across a growing heterogeneous inference infrastructure. You will drive capacity planning against real, distributed traffic and manage the operational relationship with our ASIC partner, bringing in-house expertise as the system scales.

Ideal candidates have 7+ years in infrastructure, SRE, or platform engineering within ML, HPC, or

Qualifications

  • 7+ years in infrastructure, SRE, or platform engineering, with at least some time at a serious inference, ML, or HPC shop.
  • Track record of building observability practices that catch problems, not just dashboards.
  • Background with model-serving stacks (vLLM, TGI, TensorRT-LLM, SGLang). Network fabric experience at data-center scale (RoCE, InfiniBand).

Responsibilities

  • Own the gateway and load balancer that fronts the ASIC fleet.
  • Run capacity planning against a real, distributed traffic mix across open-weight models.
  • Own the operational side of the ASIC partnership and be the technical face in partner conversations.

Skills

Infrastructure engineering
SRE experience
Platform engineering

Job description

General Compute seeks an experienced infrastructure/Platform Engineer to own the gateway and load balancer for our ASIC fleet and to oversee the observability stack across a growing heterogeneous inference infrastructure. You will drive capacity planning against real, distributed traffic and manage the operational relationship with our ASIC partner, bringing in-house expertise as the system scales.

Ideal candidates have 7+ years in infrastructure, SRE, or platform engineering within ML, HPC, or

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Head of AI Inference Infrastructure
Head of AI Inference Infrastructure

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Founding Platform Engineer: AI Inference Cloud
Founding Platform Engineer: AI Inference Cloud

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Head of Infrastructure (Remote)
Head of Infrastructure (Remote)

General Compute • San Francisco (CA)

Remote
USD 180,000 - 250,000
Platform Engineer
Platform Engineer

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Head of Inference Infrastructure
Head of Inference Infrastructure

General Compute • San Francisco (CA)

On-site
USD 190,000 - 280,000
Head of Infrastructure
Head of Infrastructure

General Compute • San Francisco (CA)

On-site
USD 190,000 - 280,000
Founding Platform Engineer
Founding Platform Engineer

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Founding Inference Engineer - AI Serving Lead
Founding Inference Engineer - AI Serving Lead

General Compute • San Francisco (CA)

On-site
USD 190,000 - 230,000
Engineering Manager, Scalable Inference Platform
Engineering Manager, Scalable Inference Platform

Anthropic Limited • San Francisco (CA)

Hybrid
USD 405,000 - 625,000
Competitive compensation
Benefits
Equity donation matching
+4
Head of Infrastructure
Head of Infrastructure

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000