Platform Engineer, Heterogeneous Inference Cloud

Black.ai

San Francisco (CA)

On-site

USD 180,000 - 260,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Black.ai in San Francisco is hiring to build the inference cloud: the control plane, API, and serving layer that turn racks into a sellable product. You will define how a mixed-hardware inference cloud gets scheduled, routed, and served reliably, in close partnership with data center teams and hardware vendors.

This early role rewards hands-on engineers who thrive on hard problems, own platform architecture, and help scale a ~6–7 person company.

Qualifications

  • Strong systems engineering background in distributed cloud control planes.
  • Proven track record building reliability from scratch.
  • Experience with resource scheduling for heterogeneous compute pools.
  • Comfort working as an early hire in a small team.
  • Familiarity with multi-cluster federation or distributed consensus systems.

Responsibilities

  • Define and own the control plane for mixed-ASIC/GPU scheduling.
  • Build API and serving layer exposing rack capacity as product.
  • Ensure reliability and observability from day one.
  • Scale platform ahead of demand and collaborate with data center teams.
  • Partner with inference engineers on model deployment strategy.

Skills

Distributed systems
Cloud infrastructure
Kubernetes-style schedulers
Reliability engineering
Resource scheduling
Heterogeneous compute
Multi-cluster federation
Software engineering

Tools

Kubernetes
Raft
ZooKeeper

Job description

Black.ai in San Francisco is hiring to build the inference cloud: the control plane, API, and serving layer that turn racks into a sellable product. You will define how a mixed-hardware inference cloud gets scheduled, routed, and served reliably, in close partnership with data center teams and hardware vendors.

This early role rewards hands-on engineers who thrive on hard problems, own platform architecture, and help scale a ~6–7 person company.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Founding Platform Engineer: AI Inference Cloud
Founding Platform Engineer: AI Inference Cloud

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Platform Engineer
Platform Engineer

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Founding Platform Engineer
Founding Platform Engineer

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Staff Software Engineer, Inference Infrastructure (Hybrid)
Staff Software Engineer, Inference Infrastructure (Hybrid)

FDE Times • San Francisco (CA), New York (NY), Seattle (WA)

Hybrid
USD 320,000 - 485,000
Competitive compensation & benefits
Equity donation matching (optional)
Generous vacation and parental leave
+2
Lead Hardware Engineer - AI Cloud Infrastructure
Lead Hardware Engineer - AI Cloud Infrastructure

Amazon Web Services (AWS) • Austin (TX)

On-site
USD 159,000 - 215,000
Health insurance
401(k) matching
Paid time off
+1
Senior Systems Engineer, AI Inference Platform
Senior Systems Engineer, AI Inference Platform

OpenAI, Inc. • San Francisco (CA)

On-site
USD 180,000 - 260,000
Staff & Senior Software Engineer, Cloud Inference
Staff & Senior Software Engineer, Cloud Inference

Anthropic Limited • San Francisco (CA)

Hybrid
USD 320,000 - 485,000
Competitive compensation
Equity donation matching
Generous vacation and parental leave
+2
Head of Inference Cloud Infrastructure
Head of Inference Cloud Infrastructure

General Compute • San Francisco (CA)

Remote
USD 180,000 - 250,000
Head of AI Inference Infrastructure
Head of AI Inference Infrastructure

Black.ai • San Francisco (CA)

On-site
USD 180,000 - 260,000
Platform Engineer for AI Inference & Optimization
Platform Engineer for AI Inference & Optimization

OpenAI, Inc. • Seattle (WA)

On-site
USD 180,000 - 240,000