AI Evaluation Engineer

Summer Browning; Greater London; Hybrid; GBP 52,000 - 76,000; Full time

Job summary

Summer Browning supports a Central Government client seeking an AI Evaluation Engineer on an initial 4 month contract. Location options include London, Bristol or Manchester with hybrid office attendance.

The role requires hands‑on development, building base code, setting up evaluation pipelines and applying AI model layers. You will define KPIs, produce technical reports and present recommendations to senior stakeholders.

Qualifications

• Proven experience in quality engineering, testing or evaluation of data‑driven and AI‑powered products.
• Strong understanding of machine learning systems, large language models (LLMs) and AI evaluation methodologies for production.
• Experience defining performance metrics, KPIs, benchmarks and acceptance criteria for AI products.
• Ability to translate business risks and policy requirements into measurable technical evaluation frameworks.
• Experience developing automated testing and evaluation pipelines using Python or similar technologies.
• Knowledge of AI safety, model robustness, bias, fairness and responsible AI principles.
• Strong analytical and problem‑solving skills with the ability to identify trends, risks and performance issues.
• Experience working with cross‑functional teams in agile product delivery environments.
• Ability to communicate complex technical findings clearly to both technical and non‑technical audiences.
• Experience producing technical reports, dashboards and recommendations for senior stakeholders.
• Familiarity with cloud‑based AI platforms and modern software development practices.
• Experience working in government, public sector or regulated environments is desirable.

Skills

Quality engineering

AI evaluation

Agile delivery

Cross-functional collaboration

Communication of findings

Data-driven evaluation frameworks

Tools

Python

Job description

SBA are currently supporting our Central Government client who are seeking an AI Evaluation Engineer on an initial 4 month contract.

Location: London, Bristol OR Manchester (hybrid office attendance)

Overview:

The team is looking for an experienced, hands‑on developer who can create their own base code, set it up and work it out. Experience applying and working with AI model layer & agentic layer

The ideal candidate:

• proven experience in quality engineering, testing or evaluation of data‑driven and AI‑powered products.
• strong understanding of machine learning systems, large language models (LLMs) and AI evaluation methodologies for production.
• experience defining performance metrics, KPIs, benchmarks and acceptance criteria for AI products.
• ability to translate business risks and policy requirements into measurable technical evaluation frameworks.
• experience developing automated testing and evaluation pipelines using Python or similar technologies.
• knowledge of AI safety, model robustness, bias, fairness and responsible AI principles.
• strong analytical and problem‑solving skills with the ability to identify trends, risks and performance issues.
• experience working with cross‑functional teams in agile product delivery environments.
• ability to communicate complex technical findings clearly to both technical and non‑technical audiences.
• experience producing technical reports, dashboards and recommendations for senior stakeholders.
• familiarity with cloud‑based AI platforms and modern software development practices.
• experience working in government, public sector or regulated environments is desirable.

NB: The successful candidate may need to undergo SC while in post, so must be eligible/willing to attain. However the successful candidate can start on BPSS