AI/ML Data Scientist — Hybrid London, Own AI Benchmarks

Medly AI; Greater London; Hybrid; GBP 65,000 - 90,000; Full time

Job summary

Medly AI in London is seeking an AI/ML Data Scientist to own how we measure and improve our AI models. You’ll work with founders and engineers who’ve built at Meta and Atlassian, designing benchmarks, evals, and experiments that deliver real improvements for students.

From day one you’ll own the work: develop subject-specific evals, improve offline test suites, run experiments, build tuning datasets, and translate pedagogical judgement into measurable signals.

Qualifications

• 2+ years in data science, ML, or applied research after graduation.
• Experience evaluating or benchmarking AI/ML systems with measurable metrics.
• Strong Python data skills (pandas, numpy) and readable code.
• Proficient SQL and relational databases.
• Solid grounding in statistics and experimental design.
• Familiarity with LLMs, evaluation, prompting, and fine-tuning.
• Degree in CS, Maths, Physics, Statistics, Engineering, Data Science, or similar.

Responsibilities

• Own AI benchmarking and create subject-specific evals for tutoring and content generation.
• Improve eval harnesses and offline test suites to ship model changes confidently.
• Run experiments and A/B tests on live AI features and interpret results.
• Build and curate high-quality datasets for fine-tuning and evaluation.
• Investigate model failures end to end and turn findings into fixes.
• Collaborate with learning developers and teachers to quantify pedagogical judgement.
• Work with product data to surface insights and shape future features.
• Contribute to the codebase (Python, Postgres, AWS, Next.js) where relevant.

Skills

Python

SQL

Statistics

Experiment design

LLMs

Eval frameworks

AWS

Education

Quantitative degree (CS, Maths, Physics, Statistics, Engineering, Data Science)

Tools

AWS

Job description

Medly AI in London is seeking an AI/ML Data Scientist to own how we measure and improve our AI models. You’ll work with founders and engineers who’ve built at Meta and Atlassian, designing benchmarks, evals, and experiments that deliver real improvements for students.

From day one you’ll own the work: develop subject-specific evals, improve offline test suites, run experiments, build tuning datasets, and translate pedagogical judgement into measurable signals.