Senior LLM Platform Engineer (Remote-Eligible)

Handshake; San Francisco (CA); Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Equity

401(k) match

Parental leave

Wellbeing stipend

Learning stipend

Remote & Office benefits

Flexible PTO

Team outings

Job summary

Handshake is hiring a Senior LLM Platform Engineer to join our Data and ML Platform team. This role owns our shared LLM control plane and related infrastructure, enabling production workflows across Handshake’s career marketplace and Handshake AI.

You will collaborate with Backend Platform, HAI engineering, and data science to build scalable, observable, and secure services, participate in on-call, and drive platform improvements from onboarding to deployment.

Qualifications

• Strong production software engineering experience in Python, TypeScript, Go, or similar language.
• Experience building or operating high-throughput API gateways, proxies, or multi-tenant platform services.
• Hands-on experience with Kubernetes, Terraform, CI/CD, and production service ownership.
• Practical experience with LLM provider APIs, including streaming, long-running requests, retries, timeouts, cancellation, and rate limits.
• Experience with authentication, quotas, credential management, tenant isolation, and auditability.
• Experience building observability for distributed systems and leading production incident response.
• Experience with usage metering, cost attribution, capacity planning, or FinOps.
• Experience with data or ML platform systems such as BigQuery, Airflow, streaming pipelines, model serving, or ML observability.
• Strong judgment in ambiguous environments and a bias toward simple, reliable systems that scale.

Responsibilities

• Build and operate LiteLLM-based AI gateways and shared LLM clients.
• Own provider and model onboarding, routing, failover, rate limits, and capacity planning.
• Build self-service workflows for keys, access, budgets, and credentials.
• Establish SLOs, observability, cost attribution, and alerts for production LLM traffic.
• Qualify and roll out new models, providers, SDKs, and gateway configurations.
• Support hosted and self-hosted inference through a consistent platform interface.
• Partner with product, AI, and FDE teams to turn recurring delivery problems into platform capabilities.
• Contribute to broader Data and ML Platform including workflow orchestration and model serving.
• Participate in team on-call and improve runbooks, automation, and reliability.

Skills

Python

TypeScript

Go

API gateways

Kubernetes

Terraform

CI/CD

LLM APIs

Authentication

Observability

FinOps

BigQuery

Airflow

Model serving

Distributed systems

Tools

LiteLLM

vLLM

Modal

Ray

PyTorch

Job description

Handshake is hiring a Senior LLM Platform Engineer to join our Data and ML Platform team. This role owns our shared LLM control plane and related infrastructure, enabling production workflows across Handshake’s career marketplace and Handshake AI.

You will collaborate with Backend Platform, HAI engineering, and data science to build scalable, observable, and secure services, participate in on-call, and drive platform improvements from onboarding to deployment.