Senior ML Engineer - AI Health Orchestration & LLM Evaluation

Doctolib (France); France; Hybrid; EUR 110,000 - 150,000; Full time

Benefits offered by this job

Free comprehensive health insurance (y

25 days paid vacation per year

RTT days

Lunch vouchers (Swile card) €8.50/day

Public transport subsidy 50%

Relocation support for international)

Job summary

Doctolib is seeking a Senior Machine Learning Engineer for the Orchestration team in the AI Health Assistant division. You will help ensure our AI Health Companion behaves safely, reliably, and helpfully for millions of patients and care teams.

You will work in a cross-functional environment to build robust evaluation pipelines for agentic AI systems, collaborating with model developers and researchers to drive iterative improvement and share findings across the company.

Qualifications

• Hold an MSc or PhD in Computer Science, Machine Learning, Data Science, or a related field.
• 7+ years of hands-on experience working with large language models.
• Experience evaluating agentic or reasoning systems (autonomous agents, tool-using LLMs, dialogue systems).
• Strong track record in experiment design, metric definition, and evaluation automation.
• Bridge research and production, influencing modeling and product decisions.
• Excellent communication skills, collaborative mindset, fluent in English.
• Healthcare domain experience and regulatory awareness is a plus.

Responsibilities

• Define and own the evaluation strategy for our AI agentic system — metrics, protocols, datasets, tooling
• Implement and maintain automated evaluation pipelines to monitor model quality, safety, and alignment
• Run systematic experiments to assess reasoning, factuality, robustness, and user experience
• Collaborate with model developers and researchers to provide insights and drive iterative improvement
• Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices

Skills

MSc or PhD in CS/ML/DS

7+ years with LLMs

Evaluate agentic/reasoning systems

Experiment design & metrics

Research to production bridge

Fluent in English

Education

MSc or PhD in CS/ML/DS

Tools

Python

Java

TypeScript

React Native

Job description

Doctolib is seeking a Senior Machine Learning Engineer for the Orchestration team in the AI Health Assistant division. You will help ensure our AI Health Companion behaves safely, reliably, and helpfully for millions of patients and care teams.

You will work in a cross-functional environment to build robust evaluation pipelines for agentic AI systems, collaborating with model developers and researchers to drive iterative improvement and share findings across the company.