Senior AI Evaluation Engineer - Public Services Impact

Incubator for Artificial Intelligence; Greater London; Hybrid; GBP 63.000 - 87.000; Vollzeit

Benefits dieser Stelle

Hybrid work from multiple cities

Generous leave

Pension contribution

Cycle to work

Dental insurance

Zusammenfassung

The Incubator for AI (i.AI) seeks two Senior AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety for frontier AI applications in public services. You will design evaluation pipelines, work with cross-functional teams, and communicate findings through reports and external publications.

Ideal candidates have strong Python skills, experience with experimental design, and a track record mentoring colleagues.

Qualifikationen

• Experience evaluating generative and predictive AI systems (LLM-as-a-judge; pydantic evals, deepeval, inspect).
• Strong knowledge of experimental design and quantitative research methods (Magenta Book).
• High proficiency in Python and software practices (git, CI/CD, unit testing, DVC or snakemake).
• Ability to communicate technical work to broad audiences via reports and presentations.
• Experience mentoring junior colleagues.

Aufgaben

• Leading the design of evaluation pipelines and product-wide evaluation suites.
• Developing methods to assess accuracy, bias, safety and model performance.
• Collaborating with product, engineering and domain experts across the lifecycle.
• Translating results into clear recommendations and communicating findings.
• Reviewing work to raise standards and build evaluation capability.

Kenntnisse

Generative AI evaluation

Python

CI/CD

Experiment design

Communication

Mentoring

Tools

pydantic evals

deepeval

inspect

DVC

Snakemake

Jobbeschreibung

The Incubator for AI (i.AI) seeks two Senior AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety for frontier AI applications in public services. You will design evaluation pipelines, work with cross-functional teams, and communicate findings through reports and external publications.

Ideal candidates have strong Python skills, experience with experimental design, and a track record mentoring colleagues.