Engineering Manager, AI Platform & Infra

Harvey; San Francisco (CA); On-site; USD 272,000 - 355,000; Full time

Job summary

Harvey is redefining professional services by combining frontier AI with an enterprise‑grade platform. As we scale, you’ll lead an engineering team building the horizontal AI platform that makes Harvey’s products faster, higher quality, and more scalable.

In this role you’ll set the technical direction, collaborate with Product, Research, and Platform teams, and grow a high‑trust organization. You’ll own delivery, code quality, and ship new AI features that solve real customer problems with

Qualifications

• 2+ years of experience in a tech‑lead/software engineering management role leading ML/AI engineering teams.
• 5+ years of experience as a backend, infrastructure or ML/AI engineer.
• A track record of leading high‑performing, engaged engineering teams.
• Experience building and operating production systems powered by LLMs (agents, retrieval, eval pipelines).
• Experience building internal platforms or shared services that other engineering teams depend on, and partnering with cross‑functional stakeholders to prioritize them.
• Strong backend engineering skills in modern languages and frameworks; familiarity with distributed systems concepts.
• Sound judgment on tradeoffs among quality, latency, cost, and speed of delivery in AI systems.
• A plus: Experience with open‑source model inference and serving, document processing, search and retrieval systems, LLM observability or working with foundation model providers.

Responsibilities

• Lead and grow a team of engineers building the core AI platform that powers Harvey’s products.
• Define the roadmap for shared AI infrastructure, including model routing, open‑source and post‑trained model serving, eval and simulation infrastructure, and large‑scale document processing, embeddings, retrieval, search and citation systems.
• Partner with Product, Research, Agent Platform, and product engineering teams to understand their needs and turn recurring patterns into reusable, well‑documented platform primitives.
• Build and scale offline eval and simulation infrastructure so teams can measure quality, catch regressions, and ship AI changes with confidence.
• Drive reliability, latency, cost efficiency, and security across LLM‑powered systems, including working with model providers on capacity and rate limits.
• Evaluate and rapidly adopt new models, frameworks, and techniques, and decide what to build, buy, or adopt.
• Contribute to code and architecture reviews, unblock teams, and lead by example when needed.
• Hire, grow, and retain exceptional engineers who thrive in a high‑trust, high‑impact environment.

Skills

Tech Lead

Backend infra

ML/AI engineering

Distributed systems

Open-source ML

LLM observability

Job description

Harvey is redefining professional services by combining frontier AI with an enterprise‑grade platform. As we scale, you’ll lead an engineering team building the horizontal AI platform that makes Harvey’s products faster, higher quality, and more scalable.

In this role you’ll set the technical direction, collaborate with Product, Research, and Platform teams, and grow a high‑trust organization. You’ll own delivery, code quality, and ship new AI features that solve real customer problems with