Head of LLM Inference Platform & Product

Modal Labs

San Francisco (CA)

On-site

USD 250,000 - 500,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Modal's LLM inference platform delivers frontier performance for open‑source models with best‑in‑class elasticity and developer experience, made in part possible by our custom runtime with GPU memory snapshots and multi‑cloud substrate.

We're looking for a leader to own the direction and execution of this platform to continue to establish us as the clear market leader, working closely with customers like Cognition, Doordash, Ramp, and many more.

Qualifications

  • 10+ years of industry experience, including 3+ years in a leadership role.
  • Track record building high‑performance systems at scale.
  • Strong background in cloud infrastructure.
  • Deep knowledge of low‑level OS foundations (Linux kernel, file systems, containers, etc.)
  • Nice to have: Experience with LLM inference production and familiarity with engines, kernels, routing, KV cache management and speculative decoding.

Responsibilities

  • Recruit, hire, and grow a high‑performing team of engineers; run regular 1:1s focused on coaching, feedback, and career growth.
  • Set clear performance expectations, hold a high bar, and build an environment where engineers do their best work.
  • Foster a culture of ownership, accountability, customer obsession, and continuous improvement.
  • Drive technical and product decisions through design reviews, code reviews, and architectural discussions.
  • Lead our efforts working with customers with frontier workloads so they can be successful running on Modal.
  • Translate learnings from frontier customers into a roadmap for our internal optimization platform and user‑facing product.
  • Establish standards for reliability and product excellence; ensure the team owns projects end‑to‑end, from spec through production.
  • Partner with business operations and compute strategy on continuing to develop our strategy for compute purchases across accelerators.
  • Collaborate with GTM on product launches, positioning, and improving win rates for inference opportunities.
  • Help guide the roadmap for teams building the infrastructure platform underlying inference, as well as adjacent product teams.

Skills

Leadership
Distributed systems
Cloud infrastructure
OS fundamentals
LLM inference (nice-to-have)

Tools

Linux
Containers
Kubernetes

Job description

Modal's LLM inference platform delivers frontier performance for open‑source models with best‑in‑class elasticity and developer experience, made in part possible by our custom runtime with GPU memory snapshots and multi‑cloud substrate.

We're looking for a leader to own the direction and execution of this platform to continue to establish us as the clear market leader, working closely with customers like Cognition, Doordash, Ramp, and many more.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

LLM Inference Platform Lead & Product Strategy
LLM Inference Platform Lead & Product Strategy

FDE Times • San Francisco (CA)

On-site
USD 240,000 - 360,000
Inference Engineering and Product Lead
Inference Engineering and Product Lead

Modal Labs • San Francisco (CA)

On-site
USD 250,000 - 500,000
Head of LLM Inference Platform & Architecture
Head of LLM Inference Platform & Architecture

Modal • San Francisco (CA)

On-site
USD 240,000 - 360,000
Inference Engineering and Product Lead
Inference Engineering and Product Lead

FDE Times • San Francisco (CA)

On-site
USD 240,000 - 360,000
Staff Research Engineer - LLM Inference & Serving
Staff Research Engineer - LLM Inference & Serving

Modal Labs • New York (NY)

On-site
USD 180,000 - 240,000
Staff Research Engineer, LLM Inference & Systems
Staff Research Engineer, LLM Inference & Systems

Fdetimes • New York (NY)

On-site
USD 180,000 - 260,000
Staff LLM Inference Research Engineer
Staff LLM Inference Research Engineer

Essence Ventures • New York (NY)

On-site
USD 180,000 - 260,000
Senior Product Manager, LLM Inference & API Growth
Senior Product Manager, LLM Inference & API Growth

Runware • United States

Remote
USD 150,000 - 210,000
Generous PTO
Meaningful stock options
Flexible hours
+2
Member of Technical Staff - Inference Research
Member of Technical Staff - Inference Research

Essence Ventures • New York (NY)

On-site
USD 180,000 - 260,000
Member of Technical Staff - Research, Inference
Member of Technical Staff - Research, Inference

Modal Labs • San Francisco (CA)

On-site
USD 180,000 - 240,000