Senior ML Engineer – AI Safety & Evaluation

Doctolib; Levallois-Perret; Hybrid; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Health insurance

Vacation days

Mental health services

Flexibility days

Lunch vouchers

Transport subsidy

Relocation support

DoctoGrowth plan

Job summary

Doctolib is seeking a Senior Machine Learning Engineer to join the Orchestration team within AI Health Assistant. You will ensure our AI Health Companion remains safe, reliable, and useful for millions of patients and care teams, building robust evaluation pipelines for agentic AI systems.

You will collaborate with model developers and researchers to drive improvements, contribute to evaluation methodologies, and help shape product decisions with strong communication and English fluency.

Qualifications

• Hold an MSc or PhD in Computer Science, ML, Data Science, or a related field.
• 7+ years of hands‑on experience with large language models (e.g., GPT, Claude, Llama, or BERT‑like architectures).
• Proven experience evaluating agentic or reasoning systems (autonomous agents, tool‑using LLMs, dialogue systems, or task‑oriented assistants).
• Strong track record in experiment design, metric definition, and evaluation automation.
• Able to bridge research and production, influencing both modeling and product decisions.
• Excellent communication skills and fluency in English.

Responsibilities

• Define and own the evaluation strategy for our AI agentic system — metrics, protocols, datasets, and tooling.
• Implement and maintain automated evaluation pipelines to monitor model quality, safety, and alignment across iterations.
• Run systematic experiments to assess reasoning, factuality, robustness, and user experience.
• Collaborate closely with model developers and research scientists to provide insights and drive iterative improvement.
• Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices.

Skills

MSc/PhD

LLMs

Experiment design

Research to production

English fluent

Communication

Education

MSc/PhD in CS/ML

Job description

Doctolib is seeking a Senior Machine Learning Engineer to join the Orchestration team within AI Health Assistant. You will ensure our AI Health Companion remains safe, reliable, and useful for millions of patients and care teams, building robust evaluation pipelines for agentic AI systems.

You will collaborate with model developers and researchers to drive improvements, contribute to evaluation methodologies, and help shape product decisions with strong communication and English fluency.