Head of Engineering

Inferact; San Francisco (CA); On-site; USD 260,000 - 380,000; Full time

Benefits offered by this job

Health, dental, vision benefits

401(k) company match

Job summary

Inferact in San Francisco is seeking a Head of Engineering to build and lead the team developing systems that power vLLM and AI inference. You’ll ensure technical credibility at the inference layer, optimizing runtimes, memory and performance across GPUs and accelerators.

You’ll partner with founders to scale a senior-heavy organization, recruit rare ML systems talent, translate ambitious work into execution plans, and deliver high-performance inference across models, hardware, and deployment

Qualifications

• Bachelor's degree or equivalent experience in CS/engineering or related field.
• Engineering leadership experience building and scaling specialized teams in LLM inference or ML systems.
• Deep technical credibility at the inference layer, including runtimes and hardware-software tradeoffs.

Responsibilities

• Lead the engineering organization developing systems powering vLLM and Inferact.
• Translate research into focused execution plans and measureable milestones.
• Recruit, coach, and retain senior engineers and staff-level ICs for production readiness.

Skills

LLM Inference

Engineering Leadership

GPU optimization

Distributed Systems

People leadership

Education

Bachelor's degree

Tools

SGLang

PyTorch

Triton

XLA

ROCm

Job description

Overview

Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware, a position that took years to build.

About the Role

We're looking for a Head of Engineering to build and lead the organization developing the systems that power vLLM and Inferact. This role requires an engineering leader with genuine technical credibility at the inference layer - someone who understands GPU and accelerator performance, inference runtimes, ML systems optimization, and hardware-software co-design deeply enough to earn the trust of exceptional staff-level engineers.

You'll partner closely with the founders to scale a senior-heavy, highly specialized engineering team while preserving the technical rigor, speed, and ownership that made vLLM successful. You'll recruit and develop rare ML systems talent, translate ambitious research and infrastructure work into a focused execution plan, strengthen how teams operate, and help Inferact deliver reliable, high-performance inference across models, hardware, and deployment environments.

Skills and Qualifications

Minimum qualifications

• Bachelor's degree or equivalent experience in computer science, engineering, machine learning, systems, or a related field.
• Engineering leadership experience building and scaling highly specialized teams in LLM inference, ML systems, GPU or accelerator software, distributed systems, or closely related infrastructure.
• Deep technical credibility at the inference layer, including hands-on understanding of inference runtimes, GPU or accelerator optimization, kernels, memory and communication bottlenecks, and hardware-software tradeoffs.
• Ability to distinguish core inference-engine work from the routing, orchestration, and application layers above it, with opinions grounded in direct technical experience.
• A strong record of recruiting, assessing, and retaining senior engineers, staff-level ICs, PhDs, and research-adjacent engineers in a production engineering environment.
• Experience translating technically ambitious work into clear priorities, accountable ownership, execution plans, and durable engineering operating mechanisms.
• Ability to remain close enough to the work to identify risks, pattern-match on difficult technical problems, and unblock teams without becoming a bottleneck or displacing technical ownership.

Preferred qualifications

• Experience leading teams responsible for LLM serving, vLLM, SGLang, model execution, inference performance, GPU kernels, compiler or runtime systems, or distributed AI infrastructure.
• Experience scaling a small, senior-heavy engineering organization where the relevant talent market is narrow and technical quality matters more than headcount growth.
• Experience integrating research-oriented or PhD talent into production teams, including setting expectations, structuring work, and building effective collaboration with product-focused engineers.
• Strong judgment across organizational design, hiring, performance management, technical planning, execution cadence, and cross-functional decision-making.
• Ability to represent the engineering organization credibly with open-source contributors, hardware partners, cloud providers, customers, candidates, and investors.
• Bonus points if you have: Built or led engineering teams working directly on GPU or accelerator-level inference performance, ML compilers, kernels, runtimes, or hardware-software co-design.
• Contributed to or led teams around open-source ML systems projects such as vLLM, SGLang, PyTorch, Ray, Triton, XLA, ROCm, or related infrastructure.
• Scaled an engineering organization through an inflection point while preserving high technical standards, fast iteration, and direct ownership.
• Recruited successfully from a global, highly competitive ML systems talent pool and built relationships with technical communities beyond traditional candidate pipelines.
• Led engineering in an early-stage AI infrastructure, developer infrastructure, distributed systems, or open-source company.

Location

This role is based in San Francisco, California. Will consider relocation for exceptional candidates.

Compensation

Compensation will be determined based on background, skills, and experience. Offer will include a highly competitive base and meaningful equity.

Visa sponsorship

We sponsor visas on a case-by-case basis.

Benefits

• Inferact offers generous health, dental, and vision benefits as well as 401(k) company match.