AI Evaluation Engineer — Hybrid, Public-Impact Tech

i.AI; City Of London; Hybrid; GBP 46,000 - 63,000; Full time

Job summary

Incubator for AI, i.AI, in the UK government ecosystem, is recruiting two AI Evaluation Engineers to design and run evaluation experiments for AI products. You will assess model accuracy, bias and safety, embedding evaluation across the product lifecycle while collaborating with product and engineering teams.

You will help create high-quality datasets, benchmarks, and clear, practical recommendations for product teams, communicating complex findings to diverse audiences.

Qualifications

• Experience in a scientific, analytic or engineering role.
• Strong analytical and critical thinking, able to derive insights from data.
• Knowledge of experimental design and statistical hypothesis testing.
• Proficient in Python and version control (e.g. git).
• Experience evaluating generative or predictive AI/ML systems.
• Experience working in a software development team and communicating technical work.

Responsibilities

• Design and build evaluation pipelines and experiments for AI products.
• Run model and system evaluations to assess accuracy, bias, safety and performance.
• Embed evaluation throughout the product lifecycle with cross-functional teams.
• Create and maintain high-quality golden datasets and benchmarks.
• Analyse results and provide actionable recommendations to product teams.
• Communicate evaluation findings to technical and non-technical audiences.
• Develop robust evaluation approaches reusable across AI products and services.

Skills

Python

Experiment design

Statistics

Git

AI/ML evaluation

Communication

Job description

Incubator for AI, i.AI, in the UK government ecosystem, is recruiting two AI Evaluation Engineers to design and run evaluation experiments for AI products. You will assess model accuracy, bias and safety, embedding evaluation across the product lifecycle while collaborating with product and engineering teams.

You will help create high-quality datasets, benchmarks, and clear, practical recommendations for product teams, communicating complex findings to diverse audiences.