Head of Engineering

LlamaIndex

San Francisco (CA)

On-site

USD 180,000 - 280,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

LlamaIndex, Inc. is seeking an experienced engineering leader to drive execution, technical direction, hiring, and team growth for its B2B AI/document understanding platform.

You will closely collaborate with leadership and product teams to decide what to build, how to build it, and how to deliver reliable software to customers. This hands-on role requires reviewing code and architecture, debugging production issues, and guiding model training, serving, and deployment strategies across services,

Qualifications

  • Leadership experience managing multiple engineering teams at a B2B SaaS scale.
  • Hands-on engineering capability with code review, architecture, and debugging.
  • Strong product judgment balancing speed, correctness, and security.
  • Clear written and spoken communication with technical teams and leadership.

Responsibilities

  • Lead and develop our engineering team, setting clear priorities, ownership, and accountability.
  • Partner with leadership to translate customer needs into a focused roadmap and deliver software predictably.
  • Stay involved in architecture, code reviews, production debugging, and implementing critical changes.
  • Set technical direction across services, APIs, data pipelines, and ML infrastructure for maintainability.
  • Collaborate with ML engineers on model training, evaluation, deployment, and production operations.
  • Guide model serving decisions considering latency, throughput, GPU usage, capacity, reliability and cost.
  • Improve engineering practices for testing, observability, security, incident response, and releases.
  • Hire strong engineers, coach leaders, and cultivate a culture of direct communication and accountability.

Job description

Join us and help shape the future of AI by defining the narrative around document understanding.

About the Role

LlamaIndex makes enterprise data useful for AI applications.

You will own engineering execution, technical direction, hiring, and team development. You will work closely with company leadership and product teams to decide what we build, how we build it, and how we deliver it reliably to customers.

This is a hands-on role. You should be comfortable reviewing code and architecture, debugging production issues, and writing code for critical systems. We are looking for someone who has shipped and operated B2B API products and understands both model serving and model training—from experimentation and evaluation to deployment and production operations.

Responsibilities
  • Lead and develop our engineering team, setting clear priorities, ownership, and accountability.
  • Partner with company leadership to translate customer needs into a focused roadmap and deliver software predictably.
  • Stay directly involved in architecture, code reviews, production debugging, and implementation of critical changes.
  • Set technical direction across application services, APIs, data pipelines, and ML infrastructure, balancing immediate customer needs with long-term maintainability.
  • Work with ML engineers and researchers on model training, fine-tuning, evaluation, and deployment, ensuring model improvements translate into better production outcomes.
  • Guide model serving decisions across latency, throughput, GPU utilization, capacity, reliability, and inference cost.
  • Improve engineering practices for testing, observability, security, incident response, and releases.
  • Hire strong engineers, coach technical leaders and managers, and build a culture of direct communication, customer focus, and responsibility for results.
Required Qualifications
  • Experience leading multiple engineering teams in B2B SaaS, at a scope comparable to a 20-30-person organization, with responsibility for delivering and operating customer-facing products.
  • Strong, current hands-on engineering skills: you can read and write production code, review architecture, and diagnose complex technical problems.
  • Practical familiarity with model serving and model training, including training or fine-tuning workflows, evaluation, deployment, and production operations.
  • Understanding of the tradeoffs between model quality, latency, throughput, GPU resources, reliability, and cost.
  • Strong background in backend systems, APIs, distributed systems, and cloud infrastructure.
  • A track record of hiring and developing engineers, setting clear expectations, and giving constructive feedback.
  • Good product judgment, including knowing when to move quickly and when to invest in correctness, reliability, and security.
  • Clear written and verbal communication with technical teams, customers, and company leadership.
Preferred Qualifications
  • Experience scaling engineering at a Series A-C startup.
  • Experience with document processing, OCR, extraction, retrieval, indexing, or other systems that work with enterprise data.
  • Experience with LLMs or multimodal models, evaluation datasets, and model quality monitoring.
  • Experience with GPU infrastructure, distributed training, inference optimization, or model serving frameworks.
  • Experience building developer tools or API-first products.
  • Experience with enterprise requirements such as multi-tenancy, access controls, auditability, and security reviews.
  • Experience developing engineering managers while staying closely involved in technical execution.

Pursuant to the SanFrancisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

LlamaIndex does not accept unsolicited agency resumes. Please do not forward resumes to our jobs alias, employees, or any other organization location. LlamaIndex is not responsible for any fees related to unsolicited resumes.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

03 Technical Lead Manager - Platform Engineering San Francisco
03 Technical Lead Manager - Platform Engineering San Francisco

LlamaIndex, Inc. • San Francisco (CA)

On-site
USD 210,000 - 330,000
Member of Technical Staff, Applied Research
Member of Technical Staff, Applied Research

LlamaIndex, Inc. • San Francisco (CA)

On-site
USD 180,000 - 250,000
Head of Engineering & ML Platform (B2B SaaS)
Head of Engineering & ML Platform (B2B SaaS)

LlamaIndex • San Francisco (CA)

On-site
USD 180,000 - 280,000
Applied AI Research Engineer - Document Understanding
Applied AI Research Engineer - Document Understanding

LlamaIndex, Inc. • San Francisco (CA)

On-site
USD 180,000 - 250,000
Senior Applied AI Engineer
Senior Applied AI Engineer

AgentGraph • United States

Remote
USD 150,000 - 190,000
401(k) match
Company HSA contributions
Wellness reimbursement
+6
Principal AI Engineer
Principal AI Engineer

VERISIGN • Reston (VA)

On-site
USD 180,000 - 260,000
Staff AI Software Engineer
Staff AI Software Engineer

Harnham • San Francisco (CA)

On-site
USD 150,000 - 200,000
Principal AI/ML Engineer
Principal AI/ML Engineer

LHH • United States

Remote
USD 180,000 - 240,000
Senior Machine Learning Engineer
Senior Machine Learning Engineer

Arionkoder • United States

Remote
USD 150,000 - 230,000
Monthly USD Service Fee
Paid Agreed Time Off
Caregiver Time Off
+2
AI Engineer
AI Engineer

AllyNd Partners • Chicago (IL)

Remote
USD 120,000 - 160,000
Annual learning stipend
Flexible hours
Comprehensive health benefits
+1