Senior ML Engineer — AI Health Safety & Evaluation

Doctolib; Paris; On-site; EUR 110,000 - 160,000; Full time

Benefits offered by this job

Health insurance (free comprehensive)

Vacation days (25 days)

RTT up to 14 days

Mental health & coaching via Moka.care

Work from abroad up to 10 days/year

Lunch vouchers (Swile €8.50/day)

Transport subsidy

DoctoGrowth plan

Relocation support

AI tools access

Job summary

Doctolib is seeking a Senior Machine Learning Engineer to join the Orchestration team within AI Health Assistant in Paris. You will help ensure the AI Health Companion behaves safely, reliably, and helpfully for millions of patients and practitioners.

You will build robust evaluation pipelines for agentic AI systems, collaborate with researchers, and contribute to improving the quality and safety of AI-powered healthcare.

Qualifications

• MSc/PhD in Computer Science, ML, Data Science, or related field.
• 7+ years of hands-on experience with large language models (e.g., GPT, Claude, Llama, BERT-like).
• Experience evaluating agentic or reasoning systems (autonomous agents, tool-using LLMs, dialogue systems).
• Strong track record in experiment design, metric definition, and evaluation automation.
• Ability to bridge research and production influencing modeling and product decisions.
• Excellent communication, collaboration, and fluency in English.

Responsibilities

• Define and own the evaluation strategy for our AI agentic system - metrics, protocols, datasets, and tooling.
• Implement and maintain automated evaluation pipelines to monitor model quality, safety, and alignment across iterations.
• Run systematic experiments to assess reasoning, factuality, robustness, and user experience.
• Collaborate closely with model developers and research scientists to provide insights and drive iterative improvement.
• Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices.

Skills

MSc/PhD in CS/ML

7+ years LLM experience

Agentic system evaluation

Experiment design

Research to production bridge

English fluency

Education

MSc or PhD in CS/ML/Data Science

Job description

Doctolib is seeking a Senior Machine Learning Engineer to join the Orchestration team within AI Health Assistant in Paris. You will help ensure the AI Health Companion behaves safely, reliably, and helpfully for millions of patients and practitioners.

You will build robust evaluation pipelines for agentic AI systems, collaborate with researchers, and contribute to improving the quality and safety of AI-powered healthcare.