Senior AI Evaluation Engineer - Hybrid UK, Public Service

Incubator for Artificial Intelligence; West of England; Hybrid; GBP 63.000 - 87.000; Vollzeit

Benefits dieser Stelle

Hybrid working locations (London/Manch

Dental insurance

Cycle to work

25+ days leave

Parental leave up to 39 weeks

Technical allowance up to £15,600

Employer pension 28.97%

Zusammenfassung

Incubator for AI (i.AI) seeks two Senior AI Evaluation Engineers to lead model-level evaluation of accuracy, bias and safety. You will design evaluation pipelines, develop advanced methods, and work with product and domain experts across London, Manchester or Bristol. Strong Python skills and mentoring experience preferred.

The role offers hybrid working, access to frontier AI, and impact on public services with outputs published through reports and external publications.

Qualifikationen

• Broad experience evaluating generative and predictive AI systems, including LLM-as-a-judge and tools such as pydantic evals, deepeval or inspect.
• Extensive knowledge of experimental design, statistical hypothesis testing and quantitative research methods (particularly those in the Magenta Book).
• High proficiency in Python and strong software practices, including git, CI/CD, unit testing and pipeline orchestration tools like DVC or snakemake.
• The ability to communicate technical work to broad audiences through reports, presentations and web-based tools.
• Experience mentoring junior colleagues.

Aufgaben

• Leading the design of evaluation pipelines and architecting product-wide evaluation suites that integrate with engineering and CI/CD workflows
• Developing complex and exploratory methods to assess accuracy, bias, safety and model performance
• Working with product, engineering and domain experts to shape evaluation across the product lifecycle and inform technical decisions
• Turning complex results into clear recommendations and leading communication of findings through reports, presentations and external publications
• Reviewing and supporting other engineers’ work to raise standards and build evaluation capability across the team

Kenntnisse

Generative AI evaluation

LLM evaluation

Python

Experimental design

Communication

Mentoring

Tools

pydantic evals

deepeval

inspect

Git

CI/CD

DVC

Snakemake

Jobbeschreibung

Incubator for AI (i.AI) seeks two Senior AI Evaluation Engineers to lead model-level evaluation of accuracy, bias and safety. You will design evaluation pipelines, develop advanced methods, and work with product and domain experts across London, Manchester or Bristol. Strong Python skills and mentoring experience preferred.

The role offers hybrid working, access to frontier AI, and impact on public services with outputs published through reports and external publications.