Senior Applied ML Engineer - LLM Routing (Remote-First)

Kimchi; Miami (FL); Remote; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Equity options

Remote-first global environment

Fast-paced delivery cycles

Job summary

Cast AI seeks an applied-ML platform engineer to own the routing model end to end—features, objective, training, calibration, and release cadence. You’ll build the evaluation that gates changes: production-parity benchmarks, router-vs-router comparisons, drift monitoring, with robust signals from live traffic.

We value strong Python and SQL at scale, grounding decisions in production fleet data and benchmarking harnesses.

Qualifications

• 5+ years in applied ML or ML research, with real-time decision models.
• Strong Python and SQL at scale, handling millions of records.
• Ability to distinguish model improvements from easier evals.
• Opinions on what LLM benchmarks capture, including agentic harnesses.
• Understanding serving realities (TTFT, decode speed, batching, concurrency).
• Instrument before you optimize, with error bars.
• A wide mandate excites you more than it unsettles you.

Responsibilities

• Own the routing model end to end: features, objective, training, calibration, release cadence.
• Build the evaluation that gates changes: production-parity benchmarks, router-vs-router comparisons, drift monitoring with error bars.
• Turn fleet data into training signals with proper failure categorization.
• Model the user-facing objective: task success × token cost × wall-clock speed under concurrency.
• Run online loops: shadow, A/B, full rollout, drift monitoring, retrain.
• Push frontier with calibrated decisions and speed-aware routing.
• Partner with platform engineers on instrumentation and data path requirements.

Skills

Applied ML

Python

SQL at scale

ML evaluation

LLM benchmarks

Deployment instrumentation

Experimentation

Tools

Python

PyTorch

SQL

ClickHouse

LLM APIs

Job description

Cast AI seeks an applied-ML platform engineer to own the routing model end to end—features, objective, training, calibration, and release cadence. You’ll build the evaluation that gates changes: production-parity benchmarks, router-vs-router comparisons, drift monitoring, with robust signals from live traffic.

We value strong Python and SQL at scale, grounding decisions in production fleet data and benchmarking harnesses.