Senior AI Evaluation Engineer — Public Sector Impact (Hybrid)

i.AI; Greater London; Hybrid; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Hybrid working

Office in London/Manchester/Bristol

Learning stipend and conference budget

Access to frontier models

Job summary

i.AI's Impact and Evaluation team is expanding to two Senior AI Evaluation Engineers who will lead evaluation pipelines across AI products, focusing on accuracy, bias and safety. You will embed with product, engineering and domain experts to shape evaluation approaches throughout the product lifecycle and translate findings into actionable recommendations for leadership.

The role offers the chance to work at scale within the government tech space, with London, Manchester or Bristol offices on a

Qualifications

• Broad experience evaluating generative and predictive AI systems.
• Excellent knowledge of experimental design and statistical testing.
• Strong Python proficiency and software development practices.

Responsibilities

• Lead the design and development of evaluation pipelines and experiments across AI products.
• Architect product-wide evaluation suites that integrate with engineering and CI/CD workflows.
• Design and implement complex evaluation methods to assess accuracy, bias, safety and model performance.
• Collaborate with product, engineering and domain experts to shape evaluation throughout the product lifecycle.
• Design and run evaluation experiments that inform product development and decisions.
• Analyse evaluation results and translate findings into recommendations for stakeholders.
• Communicate findings through reports, presentations and external publications.
• Support and review evaluation work from other engineers to raise standards.

Skills

Python

CI/CD

Experimental design

Statistical hypothesis testing

LLM evaluation

Data analysis

Communication of technical work

Tools

Git

Jupyter

Job description

i.AI's Impact and Evaluation team is expanding to two Senior AI Evaluation Engineers who will lead evaluation pipelines across AI products, focusing on accuracy, bias and safety. You will embed with product, engineering and domain experts to shape evaluation approaches throughout the product lifecycle and translate findings into actionable recommendations for leadership.

The role offers the chance to work at scale within the government tech space, with London, Manchester or Bristol offices on a