Senior ML Engineer

MyChatBot; Eu; Remote; EUR 98,000 - 143,000; Full time

Benefits offered by this job

Remote-first

Equity package

Health insurance

$2k learning budget

Compute budget for experiments

Job summary

MyChatBot is seeking an experienced ML engineer to own the model serving and orchestration layer in a remote EU setup. You will optimize latency, cost and quality across providers including OpenAI and Anthropic, and build evaluation harnesses with A/B testing to drive improvements in RAG and tool-use.

The role requires deep Python expertise, 5+ years in ML engineering, and 2+ years on production LLM systems.

Qualifications

• 5+ years of ML engineering experience.
• 2+ years production LLM systems.
• Strong Python expertise.
• Familiar with latency optimization and cost tradeoffs.

Responsibilities

• Own the model serving and orchestration layer.
• Optimize for latency, cost and quality across providers (OpenAI, Anthropic, in-house).
• Build evaluation harnesses and A/B testing infrastructure.
• Research and ship improvements to RAG, tool-use and multi-step reasoning.
• Mentor other engineers on ML best practices.

Skills

ML engineering

Python programming

LLM systems

Systems thinking

Distributed systems

Tools

vLLM

Triton

Job description

About The Role

Our agent orchestration layer routes millions of conversations a month across LLMs, voice models and custom skills. You'll own its performance, reliability and cost. We optimize for sub-300ms response times in voice and sub-2s in chat. We pick models per request based on quality vs cost tradeoffs. We A/B test prompts in production. You'll do all of this and more. This is a hands-on engineering role with significant research components. You'll write code, read papers, run experiments, and ship to production.

What you'll do

• Own the model serving and orchestration layer
• Optimize for latency, cost and quality across providers (OpenAI, Anthropic, in-house)
• Build evaluation harnesses and A/B testing infrastructure
• Research and ship improvements to RAG, tool-use and multi-step reasoning
• Mentor other engineers on ML best practices

What we’re looking for

• 5+ years ML engineering, 2+ years production LLM systems
• Deep Python; experience with vLLM, Triton or similar inference servers
• Strong systems thinking, you've debugged production latency or cost issues
• Comfortable reading research papers and prototyping their ideas
• Bonus: distributed systems experience

Perks & benefits

• Remote-first
• Equity package
• Health insurance
• $2k learning budget
• Compute budget for experiments

Department: Engineering

Location: Remote (EU)

Type: Full-time

Compensation: $110k, $160k + equity