Senior Applied ML Engineer | Kimchi (Model Router)

castaigroupinc; United States; Remote; USD 170,000 - 210,000; Full time

Benefits offered by this job

Equity options

A learning budget

Equipment support

Extra days off

Job summary

castaigroupinc is seeking an experienced applied ML professional to develop and improve the language-model routing system that decides which model answers each request. You will own the data, objectives, evaluation, and release cycle, balancing task success, speed, and cost in production-like workloads.

Responsibilities include lifecycle ownership, rigorous evaluations, and translating results into training data while accounting for infrastructure and task failures.

Qualifications

• Five+ years in applied ML or research with real-time decision models (ranking, pricing, routing, etc.).
• Strong Python and SQL skills for data extraction and analysis.

Responsibilities

• Own the routing model lifecycle from feature design to calibration and release.
• Build evaluations comparing routing approaches using production-parity benchmarks and holdouts.
• Translate outcomes and benchmarks into training data while distinguishing infrastructure vs. task failures.

Skills

Python

SQL

Job description

Role overview

Develop and improve the model that selects which language model should answer each incoming request. This applied machine-learning role owns the routing model’s data, objective, evaluation, and release cycle, with the aim of balancing task success, response speed, and cost. Decisions are validated against production-like workloads and monitored after release.

Responsibilities

• Own the routing model lifecycle, from feature design and training through calibration and release.
• Build evaluations that compare routing approaches across task types, using production-parity benchmarks, holdouts, and uncertainty estimates.
• Turn request outcomes, benchmark runs, and user overrides into training data while separating infrastructure failures from task failures.
• Define objectives that account for task success, token expense, response time, serving concurrency, and cached-input costs.
• Run shadow deployments, A/B tests, broader rollouts, and drift monitoring; investigate gaps between offline evaluation and live results.
• Work with platform engineers on request instrumentation, labels, and experiments, and explore speed-aware routing and decisions under uncertainty.

Requirements

• At least five years in applied machine learning or research, with experience training, evaluating, and shipping models for real-time decisions such as ranking, recommendations, pricing, bandits, or LLM routing.
• Strong Python and SQL skills for extracting and analyzing large volumes of request-level data.
• Sound evaluation judgment, including the ability to tell genuine model improvement from changes in benchmark difficulty.
• Understanding of LLM benchmarks and agentic evaluation, as well as serving factors such as time to first token, decoding speed, batching, caching, and concurrency.
• Habit of instrumenting systems and reporting uncertainty when measuring results.

Nice to have

Experience with LLM routing benchmarks and cost-quality-latency trade-offs, calibration or abstention methods in production, or online A/B testing with drift monitoring.

Benefits and work setup

The position is remote-first.

• Equity options
• A learning budget
• Equipment support
• Extra days off
• Time for personal development

Visa sponsorship is not available.