AI Evaluation Engineer

Summer-Browning Associates; Greater London; Hybrid; GBP 74,000 - 129,000; Full time

Job summary

Summer-Browning Associates are seeking an AI Evaluation Engineer for a four-month contract to support a Central Government client in London, Bristol or Manchester with hybrid attendance.

The role requires hands-on development, experience with AI model layers, evaluation methodologies, and the ability to translate risks into measurable frameworks. Familiarity with cloud-based AI platforms and agile delivery is essential.

Qualifications

• Proven experience in quality engineering, testing or evaluation of data-driven and AI-powered products.
• Strong understanding of ML systems, LLMs and AI evaluation methodologies for production.
• Experience defining performance metrics, KPIs, benchmarks and acceptance criteria for AI products.
• Ability to translate business risks and policy requirements into measurable technical evaluation frameworks.
• Experience developing automated testing and evaluation pipelines using Python or similar technologies.
• Knowledge of AI safety, model robustness, bias, fairness and responsible AI principles.

Responsibilities

• Create base code, set it up and work it out for AI evaluation.
• Apply and work with AI model layer & agentic layer.
• Produce technical reports, dashboards and recommendations for senior stakeholders.
• Communicate complex technical findings clearly to technical and non-technical audiences.
• Work with cross-functional teams in agile product delivery environments.
• Familiar with cloud-based AI platforms and modern software development practices.

Skills

Quality testing

AI evaluation

Metrics & KPIs

Python

AI safety & responsible AI

Agile teamwork

Communication

Data-driven evaluation

Tools

Python

Job description

SBA are currently supporting our Central Government client who are seeking an AI Evaluation Engineer on an initial 4 month contract.

Location: London, Bristol OR Manchester (hybrid office attendance)

Overview

The team is looking for an experienced, hands-on developer who can create their own base code, set it up and work it out. Experience applying and working with AI model layer & agentic layer

The ideal candidate

• proven experience in quality engineering, testing or evaluation of data-driven and AI-powered products.
• strong understanding of machine learning systems, large language models (LLMs) and AI evaluation methodologies for production.
• experience defining performance metrics, KPIs, benchmarks and acceptance criteria for AI products.
• ability to translate business risks and policy requirements into measurable technical evaluation frameworks.
• experience developing automated testing and evaluation pipelines using Python or similar technologies.
• knowledge of AI safety, model robustness, bias, fairness and responsible AI principles.
• strong analytical and problem-solving skills with the ability to identify trends, risks and performance issues.
• experience working with cross-functional teams in agile product delivery environments.
• ability to communicate complex technical findings clearly to both technical and non-technical audiences.
• experience producing technical reports, dashboards and recommendations for senior stakeholders.
• familiarity with cloud-based AI platforms and modern software development practices.
• experience working in government, public sector or regulated environments is desirable.

NB: The successful candidate may need to undergo SC while in post, so must be eligible/willing to attain. However the successful candidate can start on BPSS