Staff AI Engineer: Unified API Lead

LiteLLM; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Founders collaboration

Open source contributions

Comprehensive benefits

Job summary

LiteLLM is seeking a Staff Engineer to lead the unified API team, owning end-to-end model, endpoint, and provider rollouts. You will design provider-agnostic features at the gateway level and mentor engineers focused on LLM translation issues.

You will report to the CEO and later the Head of Engineering. This in-person role in San Francisco includes relocation and H-1B transfer support. You will shape the product's core API used by thousands of teams.

Qualifications

• You've been a machine learning engineer and now build applied AI products.
• 3+ years of experience. The scope you've owned matters more than a Staff title.
• Deep expertise in LLM APIs. You can speak to the differences between /chat/completions and /responses, and to each provider's nuances.
• Experience building or maintaining a unified API that sits in front of many providers.
• Taste in developer experience. You can tell when an API or playground is bad and explain why.
• Customer empathy. You ask why a user wants something and how big the use case is before you build it.
• Ready to lead a team from day one and set the technical bar yourself.
• Strong software engineering fundamentals in Python.

Responsibilities

• Lead the unified API team and own new model, endpoint, and provider rollouts end to end.
• Design provider-agnostic features at the gateway level, such as session management for non-OpenAI models.
• Triage open source issues and outside PRs (new providers, translation fixes) and get them merged, reviewed, or replaced with an internal fix.
• Review the AI team's PRs and hold a high bar for code quality, including code written with AI.
• Find the power users in your area and use what they need to decide what to build next.
• Take on new unification bets with a customer who will use them and give feedback.
• Own the developer experience for everyone who uses LiteLLM's Python SDK and proxy.

Skills

Machine Learning

Leadership

Python

Open APIs / LLM APIs

Developer Experience

Team Leadership

Tools

Python

FastAPI

Redis

PostgreSQL

Rust

Job description

LiteLLM is seeking a Staff Engineer to lead the unified API team, owning end-to-end model, endpoint, and provider rollouts. You will design provider-agnostic features at the gateway level and mentor engineers focused on LLM translation issues.

You will report to the CEO and later the Head of Engineering. This in-person role in San Francisco includes relocation and H-1B transfer support. You will shape the product's core API used by thousands of teams.