Senior ML Engineer: Healthcare AI Safety & Evaluation

Doctolib; Paris; Hybrid; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Health insurance

Paid vacation

Lunch vouchers

Relocation support

DoctoGrowth plan

Transport subsidy

Job summary

Doctolib is seeking a Senior Machine Learning Engineer for the Orchestration team within AI Health Assistant. You will define and own the evaluation strategy for our AI agentic system and build automated pipelines to monitor model quality, safety, and alignment across iterations.

You’ll collaborate with researchers to push iterative improvement and contribute to methodology sharing. The role requires 7+ years with large language models, MSc/PhD in a related field, and strong English

Qualifications

• MSc or PhD in Computer Science, Machine Learning, Data Science, or related field.
• 7+ years hands-on experience with large language models (e.g., GPT, Claude, Llama, or BERT-like)
• Proven experience evaluating agentic or reasoning systems (autonomous agents, tool-using LLMs, dialogue systems)
• Strong track record in experiment design, metric definition, and evaluation automation
• Ability to bridge research and production and influence modeling and product decisions

Responsibilities

• Define and own the evaluation strategy for our AI agentic system — metrics, protocols, datasets, and tooling
• Implement and maintain automated evaluation pipelines to monitor model quality, safety, and alignment across iterations
• Run systematic experiments to assess reasoning, factuality, robustness, and user experience
• Collaborate closely with model developers and research scientists to provide insights and drive iterative improvement
• Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices

Skills

LLM evaluation

Experiment design

Research-to-prod

English proficiency

Education

MSc/PhD in CS/ML/DS

Job description

Doctolib is seeking a Senior Machine Learning Engineer for the Orchestration team within AI Health Assistant. You will define and own the evaluation strategy for our AI agentic system and build automated pipelines to monitor model quality, safety, and alignment across iterations.

You’ll collaborate with researchers to push iterative improvement and contribute to methodology sharing. The role requires 7+ years with large language models, MSc/PhD in a related field, and strong English