Founding Inference Serving Engineer

Black.ai

San Francisco (CA)

On-site

USD 190,000 - 240,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

General Compute is building a specialized inference serving stack for dedicated silicon, and this founding role will own the end-to-end serving pipeline. You will design and implement a reliable, low-latency system that handles real traffic at scale and interfaces with the compiler and bring-up teams.

The role requires 5+ years in production infrastructure, deep knowledge of LLM inference serving, and a bias toward reliability.

Qualifications

  • 5+ years building and operating production systems at the infrastructure layer.
  • Direct experience with LLM inference serving in a production environment.
  • Comfortable owning reliability and designing for failure.

Responsibilities

  • Own the inference serving stack end-to-end from model bring-up to production.
  • Tune batching, KV-cache handling, and hardware utilization for throughput.
  • Implement monitoring, alerting, and failover for reliability under load.
  • Define interfaces with compiler and bring-up teams and shape the serving roadmap.
  • Set technical standards to guide a growing serving team.

Skills

Production systems
LLM inference
Reliability on-call
Systems fundamentals
Ambiguity tolerance

Job description

General Compute is building a specialized inference serving stack for dedicated silicon, and this founding role will own the end-to-end serving pipeline. You will design and implement a reliable, low-latency system that handles real traffic at scale and interfaces with the compiler and bring-up teams.

The role requires 5+ years in production infrastructure, deep knowledge of LLM inference serving, and a bias toward reliability.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Founding Inference Engineer - AI Serving Lead
Founding Inference Engineer - AI Serving Lead

General Compute • San Francisco (CA)

On-site
USD 190,000 - 230,000
Founding Inference Engineer
Founding Inference Engineer

General Compute • San Francisco (CA)

On-site
USD 190,000 - 230,000
Inference Engineer
Inference Engineer

Black.ai • San Francisco (CA)

On-site
USD 190,000 - 240,000
Founding Platform Engineer: AI Inference Cloud
Founding Platform Engineer: AI Inference Cloud

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Founding Platform Engineer
Founding Platform Engineer

General Compute • San Francisco (CA)

On-site
USD 240,000 - 300,000
Head of Inference Cloud Infrastructure
Head of Inference Cloud Infrastructure

General Compute • San Francisco (CA)

Remote
USD 180,000 - 250,000
Head of Infrastructure (Remote)
Head of Infrastructure (Remote)

General Compute • San Francisco (CA)

Remote
USD 180,000 - 250,000
Inference Engineer
Inference Engineer

Acceler8 Talent • Palo Alto (CA)

On-site
USD 180,000 - 320,000
Founding Inference Systems Engineer
Founding Inference Systems Engineer

Touchdown Labs • United States

Remote
USD 140,000 - 210,000
LLM Inference Engineer: Peak-Performance AI Serving
LLM Inference Engineer: Peak-Performance AI Serving

Majestic Labs ai • Los Altos (CA)

On-site
USD 210,000 - 350,000