Senior AI Evaluation Engineer (Data Scientist)

i.AI; Greater London; Hybrid; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Hybrid working

Office in London/Manchester/Bristol

Learning stipend and conference budget

Access to frontier models

Job summary

i.AI's Impact and Evaluation team is expanding to two Senior AI Evaluation Engineers who will lead evaluation pipelines across AI products, focusing on accuracy, bias and safety. You will embed with product, engineering and domain experts to shape evaluation approaches throughout the product lifecycle and translate findings into actionable recommendations for leadership.

The role offers the chance to work at scale within the government tech space, with London, Manchester or Bristol offices on a

Qualifications

• Broad experience evaluating generative and predictive AI systems.
• Excellent knowledge of experimental design and statistical testing.
• Strong Python proficiency and software development practices.

Responsibilities

• Lead the design and development of evaluation pipelines and experiments across AI products.
• Architect product-wide evaluation suites that integrate with engineering and CI/CD workflows.
• Design and implement complex evaluation methods to assess accuracy, bias, safety and model performance.
• Collaborate with product, engineering and domain experts to shape evaluation throughout the product lifecycle.
• Design and run evaluation experiments that inform product development and decisions.
• Analyse evaluation results and translate findings into recommendations for stakeholders.
• Communicate findings through reports, presentations and external publications.
• Support and review evaluation work from other engineers to raise standards.

Skills

Python

CI/CD

Experimental design

Statistical hypothesis testing

LLM evaluation

Data analysis

Communication of technical work

Tools

Git

Jupyter

Job description

Applications for this position close on 3rd November 2026 - 23.45pm UK Time

Incubator for AI, i.AI

Help shape how people interact with some of the most ambitious AI products in government

The Incubator for AI (i.AI) is a team of ~80 builders and operators using frontier AI to improve public services across healthcare, education, planning and parliament. We combine startup pace and autonomy with the opportunity to tackle national-scale challenges.

We operate under three core principles:

Talent : We bring together the UK's best AI talent across a range of functions. You will work alongside exceptional researchers and top government leaders, staying at the cutting edge of technology.

Innovation : We set precedents for what is possible in government. We combine the pace of a start-up with the influence of being at the digital centre of government. You will test new ideas, expand what is possible, and leverage unique government data to create novel solutions.

Impact : We are dedicated to using AI as a tool for public good - this can mean improving outcomes in schools, boosting housebuilding or providing more personalised support for those in need. With the backing of the Prime Minister, you will turn technical breakthroughs into real-world applications that affect millions of citizens.

About the role

i.AI's Impact and Evaluation team build their work into each stage of product development, from scoping, to incubation, and onwards to scaling. Team members embed into product development teams, designing and delivering robust evaluations at each stage of the product journey to inform their design and, ultimately, understand their impact on the quality and efficiency of public service delivery.

The team is now recruiting for two Senior AI Evaluation Engineers, to lead the evaluation of AI products. These roles will focus on model-level evaluation, including assessment of accuracy, bias and safety.

What you will work on

• Lead the design and development of evaluation pipelines and experiments across AI products
• Architect product-wide evaluation suites that integrate with engineering and CI/CD workflows
• Design and implement complex and exploratory evaluation methods to assess accuracy, bias, safety and overall model performance
• Work closely with product, engineering and domain experts to shape evaluation approaches throughout the product lifecycle
• Design and run evaluation experiments that directly inform product development and technical decisions
• Analyse complex evaluation results and turn findings into clear recommendations for technical, product and senior stakeholders
• Lead the communication of evaluation findings through reports, presentations and external publications
• Support and review evaluation work from other engineers, helping to raise standards and develop evaluation capability across the team

Who we think will thrive in this role

You may be a strong fit if you bring some of the following:

• Broad experience of evaluating generative and predictive AI systems
• Extensive knowledge of experimental design and statistical hypothesis testing
• Highly proficient in Python
• Demonstrable experience of software development tools and practices (e.g. git, CI/CD, unit testing)
• Ability to communicate technical work to a broad audience, using different media/formats e.g. technical reports, presentations, and web-based tools.
• Experience implementing LLM-as-a-judge and other generative AI evaluation techniques using AI evaluation tools (e.g. pydantic evals, deepeval, inspect)
• Experience using pipeline orchestration tools e.g. DVC, snakemake
• Knowledge of standard quantitative research designs, methods and analysis (particularly those set out in the Magenta Book)
• Experience of mentoring junior colleagues

What we offer

• Career-defining projects with outsized impact
• Backing from the Prime Minister and No10 to scope and build transformative AI projects.
• Unique opportunities to apply technology that could transform the public sector and impact citizens' lives.
• Talented, supportive and mission-driven colleagues.

Resources & access

• Access to frontier models and ample compute.
• Extensive operational, product, strategy, design and delivery support so you can focus on shipping.
• Work with experts across national security, policy, AI research and adjacent sciences.

Growth & empowerment

• A team culture and development support that prioritises personal growth.
• Opportunities to own important products early and develop them in small empowered teams.
• 5 days off learning and development, annual stipends for learning and development and funding for conferences and external collaborations.

Life & family*

• Opportunity to work from London, Manchester or Bristol offices.
• Hybrid working, flexibility for occasional remote work abroad and stipends for work-from-home equipment.
• Generous annual leave - 25 days plus one additional day for each year of service
• Generous paid parental leave (up to 39 weeks full pay + op