Senior ML Engineer: Low-Latency Orchestration (Remote)

MyChatBot; Eu; Remote; EUR 98,000 - 143,000; Full time

Benefits offered by this job

Remote-first

Equity package

Health insurance

$2k learning budget

Compute budget for experiments

Job summary

MyChatBot is seeking an experienced ML engineer to own the model serving and orchestration layer in a remote EU setup. You will optimize latency, cost and quality across providers including OpenAI and Anthropic, and build evaluation harnesses with A/B testing to drive improvements in RAG and tool-use.

The role requires deep Python expertise, 5+ years in ML engineering, and 2+ years on production LLM systems.

Qualifications

• 5+ years of ML engineering experience.
• 2+ years production LLM systems.
• Strong Python expertise.
• Familiar with latency optimization and cost tradeoffs.

Responsibilities

• Own the model serving and orchestration layer.
• Optimize for latency, cost and quality across providers (OpenAI, Anthropic, in-house).
• Build evaluation harnesses and A/B testing infrastructure.
• Research and ship improvements to RAG, tool-use and multi-step reasoning.
• Mentor other engineers on ML best practices.

Skills

ML engineering

Python programming

LLM systems

Systems thinking

Distributed systems

Tools

vLLM

Triton

Job description

MyChatBot is seeking an experienced ML engineer to own the model serving and orchestration layer in a remote EU setup. You will optimize latency, cost and quality across providers including OpenAI and Anthropic, and build evaluation harnesses with A/B testing to drive improvements in RAG and tool-use.

The role requires deep Python expertise, 5+ years in ML engineering, and 2+ years on production LLM systems.