Quality Systems Lead

present of AI; Greater London; Hybrid; GBP 75,000 - 110,000; Full time

Job summary

Encord in London seeks a Senior Operations professional to own the quality standard for human-labeled data. You will oversee both automated evaluation systems and a distributed audit team, writing rubrics and setting processes that scale with remote work.

You should be hands-on in Python and SQL, comfortable with sampling and agreement statistics, and able to defend standards under commercial pressure with clear evidence.

Qualifications

• Hands-on experience with Python and SQL.
• Ability to write clear rubrics for a remote team to follow.
• Experience with sampling and agreement statistics under commercial pressure.

Responsibilities

• Own the quality standard for human-labeled data, spanning automated evaluation and a distributed audit team.
• Oversee model-based evaluation and maintain rigorous quality controls under business constraints.
• Provide evidence-backed justification for standards and maintain traceability of decisions.

Skills

Python

SQL

Sampling

Agreement statistics

Job description

Operations · London · Full-time · Senior level

This role owns the quality standard for the human labelled data the company delivers, and runs both halves of it: automated evaluation built with models and statistics, and a distributed audit team of about ten people. A strong candidate is hands on in Python and SQL, comfortable with sampling and agreement statistics, and able to write rubrics clear enough for a remote team to follow without supervision. The job also asks for someone who can hold a standard under commercial pressure and show the evidence behind it.

How it fits the AI era

Much of the screening work here is meant to be done by models, with LLM based judging and anomaly detection replacing linear growth in headcount. What stays human is the judgement on hard cases, the calibration of the standard, and the decision about how much manual coverage to keep. The role sits at the point where automated measurement and human ground truth feed each other.

How it builds AI experience

The work offers direct practice in LLM as judge screening, model assisted QA and drift detection on real production annotation data, plus building gold sets that train and validate those models. Because the company owns the platform, the measurement built here can become product capability rather than internal tooling, which is a rare chance to shape evaluation features for physical AI customers.

Worth showing

• Python and SQL
• LLM as judge evaluation
• Sampling and agreement statistics
• Writing quality rubrics

This is our own read, written from the employer's posting and the company's Spotlight. The employer's posting is the source of truth for the role.

About Encord

A data platform that helps AI teams manage, label and clean the video, sensor and image data that trains robots, self-driving cars and other physical AI systems.