AI Evaluation Engineer

Incubator for Artificial Intelligence; West of England; Hybrid; GBP 46,000 - 62,000; Full time

Benefits offered by this job

Hybrid working (London, Manchester, or

25+ days’ leave

Dental insurance

Employer pension contribution

Job summary

Incubator for AI in the UK is hiring two AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety, across London, Manchester and Bristol.

You will design evaluation pipelines, run model and system evaluations, and translate findings into practical recommendations for product teams and public services.

You'll collaborate with domain experts, communicate results to technical and non-technical audiences, and help scale evaluation as products grow.

Qualifications

• Experience in a scientific, analytical or engineering role.
• Strong analytical and critical thinking with data insight.
• Knowledge of experimental design and statistical testing.
• Proficiency in Python and version control (git).
• Experience evaluating AI/ML systems and communicating findings.

Responsibilities

• Design and build evaluation pipelines and experiments for AI products.
• Run model and system evaluations for accuracy, bias and safety.
• Collaborate with product and engineering to embed evaluation from day one.
• Partner with domain experts to create golden datasets and benchmarks.
• Translate evaluation outcomes into practical recommendations for teams.
• Explain results to both technical and non-technical audiences.
• Develop reusable evaluation approaches applicable across products.

Skills

Python

Git

Experimental design

Statistical hypothesis testing

Data analysis

Communication of results

Tools

Pandas

scikit-learn

Jupyter

Job description

AI Evaluation Engineer (Data Scientist) - London/Manchester/Bristol

The Incubator for AI (i.AI) is a team of around 80 builders and operators using frontier AI to improve public services across healthcare, education, planning and parliament. We work at startup pace, with the backing of the Prime Minister and No10, on problems that affect millions of people.

Our Impact and Evaluation team embeds within product teams to build rigorous evaluation into every stage of development, shaping product decisions and showing whether our products improve public services. We’re now hiring two AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety.

What will I be doing?

• Designing and building evaluation pipelines and experiments for AI products
• Running model and system evaluations to test accuracy, bias, safety and overall performance
• Working with product and engineering teams to build evaluation in from day one through to scale
• Partnering with domain experts to create and maintain high-quality golden datasets and benchmarks
• Turning evaluation results into clear, practical recommendations for product teams
• Helping teams understand the real-world impact of their products on public service delivery
• Communicating findings to technical and non-technical audiences through reports and presentations
• Developing reusable evaluation approaches that work across many products and services

What will I need?

You don’t need to tick every box. You’ll likely bring some of the following:

• Experience in a scientific, analytical or engineering role
• Strong analytical and critical thinking skills, and the ability to draw insight from complex data
• Practical knowledge of experimental design and statistical hypothesis testing
• Proficiency in Python and version control (e.g. git)
• Experience evaluating generative or predictive AI/ML systems
• Experience working in a software development team
• The ability to explain technical work clearly to a broad audience

What’s in it for me?

• Projects backed by No10 that apply AI to public services affecting millions, alongside talented, mission‑driven colleagues
• Frontier models, ample compute, access to experts in national security, policy and AI research, and full operational and delivery support
• Early ownership of important products in small teams, 5 L&D days a year, and funding for learning, conferences and collaborations
• £48,540–£62,970 in London (£45,740–£59,835 nationally), including a technical allowance of up to £9,000, plus a 28.97% employer pension contribution
• Flexibility and benefits. Hybrid working from London, Manchester or Bristol, 25+ days’ leave, up to 39 weeks’ full‑pay parental leave, and perks including cycle to work and dental insurance