Senior Applied ML Engineer — Real-Time LLM Router

Cast AI; France; Remote; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Remote-first global environment

Equity options

Learning budget

Equipment budget

Job summary

Cast AI is hiring for an applied ML Platform Engineer to own the routing model end to end. You will define features, objectives, training, calibration, and release cadence, ensuring production-parity benchmarks and robust evaluation with error bars.

You will transform per-request outcomes and benchmark rollouts into training signals, balancing task success with token cost and decoding speed. This is a remote, globally distributed role with strong emphasis on instrumentation, experimentation, and

Qualifications

• You have 5+ years in applied ML or ML research with real-time decision experience.
• Strong Python and SQL at scale; able to interrogate millions of records.
• Understand model quality, cost, latency trade-offs and how to measure them.
• Experience with LLM routing, benchmarks, and online experimentation is a plus.
• Ability to instrument systems and reason about conflicting signals.

Responsibilities

• Own the routing model end to end: features, objective, training, calibration, release cadence.
• Build and maintain production-parity evaluation with error bars andholdouts.
• Turn fleet data into training signals while managing data quality.
• Model the objective from user perspective: task success, token cost, and speed.
• Run shadow, A/B, and full rollout with drift monitoring and retraining.

Skills

Applied ML experience

Python

SQL at scale

Model evaluation

A/B testing / online experiments

Communication of metrics

Tools

PyTorch

ClickHouse

OpenAI API

Anthropic API

RouterBench

GitLab CI

Python

Job description

Cast AI is hiring for an applied ML Platform Engineer to own the routing model end to end. You will define features, objectives, training, calibration, and release cadence, ensuring production-parity benchmarks and robust evaluation with error bars.

You will transform per-request outcomes and benchmark rollouts into training signals, balancing task success with token cost and decoding speed. This is a remote, globally distributed role with strong emphasis on instrumentation, experimentation, and