Data Labeling Lead for AI Safety & RLHF

Moonfire; Greater London; Hybrid; GBP 60.000 - 100.000; Vollzeit

Benefits dieser Stelle

Equity

Flexible time off

Office in Paris

Relocation support

Private health insurance

Mental health support

Language lessons (English/French)

Lunch and dinner provided at office

Learning & development

All hardware and tools

Team off-sites

Zusammenfassung

White Circle is seeking a Data Labeling Lead to build and lead our annotation team from the ground up. You will define guidelines, quality standards, and evaluation frameworks to support safety evals, RLHF, and benchmarks.

You'll own quality, throughput, and cost while partnering with AI researchers to translate research objectives into labeling workflows and drive scalable operations.

Qualifikationen

• Experience leading data annotation or AI evaluation teams.
• Strong operational and people management skills.
• Understanding AI model evaluation, LLM behaviour, and modern annotation workflows.

Aufgaben

• Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management).
• Define annotation guidelines, quality standards, and evaluation frameworks.
• Develop quality assurance processes, calibration sessions, and auditing systems.
• Partner with AI researchers and engineers to translate research objectives into labeling workflows.
• Prioritise labeling projects based on business and research needs.
• Monitor operational metrics including quality, consistency, throughput, and cost.
• Improve annotation tooling, automation, and workflow efficiency.
• Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation.
• Analyse disagreement patterns and edge cases to improve guidelines and model performance.
• Manage vendor relationships and ensure consistent quality across distributed teams.
• Build reporting dashboards and communicate operational insights to leadership.
• Foster a culture of continuous improvement, accountability, and operational excellence.

Kenntnisse

Data labeling

Team leadership

Operations management

AI evaluation

Communication

Tools

Annotation tools

Workflow automation

Jobbeschreibung

White Circle is seeking a Data Labeling Lead to build and lead our annotation team from the ground up. You will define guidelines, quality standards, and evaluation frameworks to support safety evals, RLHF, and benchmarks.

You'll own quality, throughput, and cost while partnering with AI researchers to translate research objectives into labeling workflows and drive scalable operations.