AI Evaluation Quality Lead

Inquigent Ltd.; United Kingdom; Remote; GBP 65,000 - 95,000; Full time

Benefits offered by this job

Remote-first

Learning budget

Equipment provided

Job summary

Inquigent Ltd. is seeking an AI Evaluation Quality Lead to own and uphold the quality standards behind assessment tasks, answer keys, reviewer guidance, and feedback for managed evaluation projects.

You will work hands-on in LLM evaluation, data annotation quality, and related areas to ensure consistent, defensible decisions. You will design rubrics, calibrate reviewers, review advisory outputs, and develop actionable guidance.

Qualifications

• Hands-on experience in LLM evaluation, data annotation quality, search or ads quality rating, or similar.
• Ability to write a rubric that two reviewers apply the same way.
• Analytical and precise; comfortable with sampling and simple metrics.

Responsibilities

• Design and maintain assessment content, rubrics and reviewer guidance so decisions are consistent and defensible
• Set and check quality standards on managed projects: sampling, calibration and feedback loops
• Review AI advisory grading output and decide where it helps and where it misleads
• Write development guidance members can act on after a retry decision
• Work with the Talent Operations Lead on reviewer training and calibration sessions

Skills

LLM evaluation experience

Data annotation quality

Analytical and precise

Sampling and metrics

Rubric writing

Job description

Our practical assessment and our managed projects only work if the standards behind them are excellent. The AI Evaluation Quality Lead owns those standards: the assessment tasks and answer keys, the guidance reviewers follow, the quality bar on managed evaluation and annotation projects, and the feedback members receive. You will have worked hands‑on in LLM evaluation, data annotation or search quality and know what good looks like.

What you'll do

• Design and maintain assessment content, rubrics and reviewer guidance so decisions are consistent and defensible
• Set and check quality standards on managed projects: sampling, calibration and feedback loops
• Review AI advisory grading output and decide where it helps and where it misleads
• Write development guidance members can act on after a retry decision
• Work with the Talent Operations Lead on reviewer training and calibration sessions

What we're looking for

• Hands‑on experience in LLM evaluation, data annotation quality, search or ads quality rating, or similar
• You can write a rubric that two reviewers apply the same way
• Analytical and precise; comfortable with sampling and simple metrics
• Right to work in the United Kingdom or Nigeria

Nice to have

• Experience training or calibrating raters at scale
• Familiarity with responsible‑AI evaluation practice
• Experience with multilingual evaluation

What we offer

• Remote‑first with flexible hours across UK and West Africa time zones
• Real ownership of standards that affect how people are assessed
• Equipment provided and a learning budget