Senior LLM Platform Engineer - AI Gateways & Infra

Fdetimes; San Francisco (CA); On-site; USD 180,000 - 280,000; Full time

Benefits offered by this job

Equity in a fast-growing company

401(k) match & financial coaching

Medical, dental, and vision benefits

Mental health support

$2,000 learning stipend

Flexible PTO and holidays

Team outings & referral bonuses

Free lunch/gym in SF office

Job summary

Handshake is hiring a Senior LLM Platform Engineer to join our Data and ML Platform team. You will build and operate LiteLLM gateways and shared LLM clients, own provider onboarding, routing, and capacity planning, and contribute to orchestration, model serving, and developer tooling.

You will collaborate closely with Backend Platform, HAI engineering, data science, and FDEs to deliver scalable, reliable AI infrastructure across production workflows.

Qualifications

• Strong production software engineering experience in Python, TypeScript, Go.

Responsibilities

• Build and operate LiteLLM-based AI gateways and shared LLM clients.
• Own provider onboarding, routing, failover, rate limits, and capacity planning.
• Develop self-service workflows for model access, budgets, and credentials.
• Establish SLOs, observability, cost attribution, and alerts for LLM traffic.
• Safely qualify and roll out new models, providers, and gateway configurations.
• Support hosted and self-hosted inference through a unified platform interface.
• Collaborate with product, AI, and FDE teams to improve platform capabilities.
• Contribute to broader Data and ML Platform including orchestration and model serving.
• Participate in on-call duties and improve runbooks and reliability.

Skills

Python

TypeScript

Go

Kubernetes

Terraform

CI/CD

LLM APIs

Security & auth

Observability

FinOps

Tools

LiteLLM

FastAPI

PostgreSQL

Redis

GCP

Kubernetes

Terraform

Spacelift

BigQuery

Airflow

OpenAI

Anthropic

Gemini

vLLM

Modal

Ray

Job description

Handshake is hiring a Senior LLM Platform Engineer to join our Data and ML Platform team. You will build and operate LiteLLM gateways and shared LLM clients, own provider onboarding, routing, and capacity planning, and contribute to orchestration, model serving, and developer tooling.

You will collaborate closely with Backend Platform, HAI engineering, data science, and FDEs to deliver scalable, reliable AI infrastructure across production workflows.