AI Evaluation Engineer | Hybrid & Public Service Impact

Incubator for Artificial Intelligence; Greater London; Hybrid; GBP 49,000 - 63,000; Full time

Benefits offered by this job

Cycle to work

Dental insurance

Parental leave (39 weeks)

Job summary

The Incubator for AI (i.AI) is hiring two AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety, embedded within product teams. You will design evaluation pipelines and experiments for AI products, run model and system tests, and partner with domain experts to build high-quality golden datasets and benchmarks.

You’ll translate findings into practical recommendations for product teams and help teams understand real-world impact on public services.

Qualifications

• Experience in a scientific, analytical or engineering role.
• Ability to draw insight from complex data.
• Practical knowledge of experimental design and statistical hypothesis testing.
• Proficiency in Python and version control (e.g. git).
• Experience evaluating generative or predictive AI/ML systems.
• Ability to explain technical work clearly to a broad audience.
• Experience working in a software development team.

Responsibilities

• Designing and building evaluation pipelines and experiments for AI products.
• Running model and system evaluations to test accuracy, bias, safety and overall performance.
• Working with product and engineering teams to build evaluation in from day one through to scale.
• Partnering with domain experts to create and maintain high-quality golden datasets and benchmarks.
• Turning evaluation results into clear, practical recommendations for product teams.
• Helping teams understand the real-world impact of their products on public service delivery.
• Communicating findings to technical and non-technical audiences through reports and presentations.
• Developing reusable evaluation approaches that work across many products and services.

Skills

Analytical thinking

Experimental design

Statistical testing

Python

Git

AI/ML evaluation

Clear communication

Tools

Git

Job description

The Incubator for AI (i.AI) is hiring two AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety, embedded within product teams. You will design evaluation pipelines and experiments for AI products, run model and system tests, and partner with domain experts to build high-quality golden datasets and benchmarks.

You’ll translate findings into practical recommendations for product teams and help teams understand real-world impact on public services.