Senior LLM Platform Engineer — Scalable Gateways

FDE Times; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Equity

401(k) Match

Paid Parental Leave

Fertility Benefits

Medical Insurance

Dental Insurance

Vision Insurance

Mental Health Support

Wellness Stipend

Learning Stipend

Internet Allowance

Commuting Benefits

Free Lunch

Gym Access

Flexible Paid Time Off

Paid Holidays

Team Outings

Referral Bonuses

Job summary

FDE Times is seeking a senior software engineer to build and operate scalable LLM infrastructure in a multi-tenant environment. You will work on LiteLLM gateways, provider integrations, routing, access controls, and observability, with on-call responsibilities to ensure reliability.

You will collaborate with engineering, data science, and product teams to improve developer enablement and platform robustness while focusing on scalability, cost attribution, and capacity planning.

Qualifications

• Strong production software engineering experience.
• Hands-on knowledge of high-throughput gateways or multi-tenant services.
• Practical experience with LLM APIs, access/credential management, distributed observability, incident response, and usage metering/capacity planning.

Responsibilities

• Build and operate shared LLM infrastructure and gateways.
• Partner with engineering, data science, product, and FDE teams.
• Provide on-call support and improve reliability and developer enablement.

Skills

Kubernetes

Terraform

CI/CD

LLM Platform Engineering

API Gateways

Observability

Incident Response

Cost Attribution

Capacity Planning

Model Serving

Workflow Orchestration

Python

TypeScript

Go

LLM Provider APIs

Authentication and Access Controls

Tools

Job description

FDE Times is seeking a senior software engineer to build and operate scalable LLM infrastructure in a multi-tenant environment. You will work on LiteLLM gateways, provider integrations, routing, access controls, and observability, with on-call responsibilities to ensure reliability.

You will collaborate with engineering, data science, and product teams to improve developer enablement and platform robustness while focusing on scalability, cost attribution, and capacity planning.