Head of AI Inference Infrastructure

Black.ai

San Francisco (CA)

On-site

USD 180,000 - 260,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

General Compute is seeking an experienced Infrastructure, SRE, or Platform Engineer to own the inference cloud’s control plane and gateway. You will lead the observability stack, capacity planning, and vendor relationships with ASIC partners as we build a heterogeneous fleet of accelerators and GPUs.

The first six months are hands-on with Kubernetes manifest work, dashboards, on-call, and collaboration with our ASIC partner’s engineering team.

Qualifications

  • 7+ years in infrastructure, SRE, or platform engineering, with ML/AI inference or HPC exposure.
  • Hands-on Kubernetes at production scale, including debugging complex issues.
  • Strong focus on tail latency and p99 utilization.

Responsibilities

  • Own the inference control plane and evolve it beyond current configuration.
  • Own gateway/load balancer for the ASIC fleet, including model placement and routing.
  • Own observability end-to-end with per-request tracing and SLO dashboards.
  • Run capacity planning across distributed traffic and open-weight models.
  • Manage the ASIC partnership and be the technical liaison.
  • Bring up the pre-fill side on a new hardware platform.
  • Build the on-call and incident response program and hire the team.

Skills

Kubernetes operations
Tail latency optimization
SRE / platform engineering
On-call incident response
Observability practices
Vendor management
Infra leadership potential

Job description

General Compute is seeking an experienced Infrastructure, SRE, or Platform Engineer to own the inference cloud’s control plane and gateway. You will lead the observability stack, capacity planning, and vendor relationships with ASIC partners as we build a heterogeneous fleet of accelerators and GPUs.

The first six months are hands-on with Kubernetes manifest work, dashboards, on-call, and collaboration with our ASIC partner’s engineering team.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Head of Inference Cloud Infrastructure
Head of Inference Cloud Infrastructure

General Compute • San Francisco (CA)

Remote
USD 180,000 - 250,000
Founding Platform Engineer: AI Inference Cloud
Founding Platform Engineer: AI Inference Cloud

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Head of Infrastructure (Remote)
Head of Infrastructure (Remote)

General Compute • San Francisco (CA)

Remote
USD 180,000 - 250,000
Founding Inference Engineer - AI Serving Lead
Founding Inference Engineer - AI Serving Lead

General Compute • San Francisco (CA)

On-site
USD 190,000 - 230,000
Remote GPU Inference Infra Platform Engineer
Remote GPU Inference Infra Platform Engineer

Together Computer Inc • United States

Remote
USD 180,000 - 240,000
Platform Engineer
Platform Engineer

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Head of Inference Infrastructure
Head of Inference Infrastructure

General Compute • San Francisco (CA)

On-site
USD 190,000 - 280,000
Senior AI Inference Reliability Engineer - GPU Compute Infra
Senior AI Inference Reliability Engineer - GPU Compute Infra

Together Computer Inc • San Francisco (CA)

On-site
USD 250,000 - 320,000
Equity
Health insurance
Competitive benefits
Head of AI Inference & Systems Platform
Head of AI Inference & Systems Platform

Blackhornvc • San Francisco (CA)

On-site
USD 220,000 - 420,000
Head of Infrastructure
Head of Infrastructure

General Compute • San Francisco (CA)

On-site
USD 190,000 - 280,000