Head of Data Labeling & AI Evaluation

Visa Hunt; Paris; Hybride; EUR 65 000 - 90 000; Plein temps

Avantages offerts par ce poste

Competitive salary + equity

Hybrid work from Paris with relocation

London option (limited relocation/ins)

Paid time off

Hardware, tools & subscriptions

Team off-sites twice a year

Résumé du poste

White Circle is seeking a Data Labeling Lead to build and guide a scalable labeling program within a safety-focused AI environment. You will hire and coach a distributed team, set guidelines, and drive QA, calibration, and auditing processes in partnership with AI researchers and engineers.

You will manage vendor relationships, monitor quality and cost, and push for tooling enhancements. This role requires strong people skills and a startup mindset, with hybrid work options from Paris or London.

Qualifications

• Experience leading data annotation or AI evaluation teams.
• Strong operational and people management skills.
• Understanding AI model evaluation, LLM behavior, and modern annotation workflows.
• Ability to design scalable processes without sacrificing quality.
• Clear communication across technical and non-technical teams.
• Thrives in fast-moving startup environments.

Responsabilités

• Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management).
• Define annotation guidelines, quality standards, and evaluation frameworks.
• Develop quality assurance processes, calibration sessions, and auditing systems.
• Partner with AI researchers and engineers to translate research objectives into labeling workflows.
• Prioritize labeling projects based on business and research needs.
• Monitor operational metrics including quality, consistency, throughput, and cost.
• Improve annotation tooling, automation, and workflow efficiency.
• Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation.
• Analyse disagreement patterns and edge cases to improve guidelines and model performance.
• Manage vendor relationships and ensure consistent quality across distributed teams.
• Build reporting dashboards and communicate operational insights to leadership.
• Foster a culture of continuous improvement, accountability, and operational excellence.

Connaissances

Data labeling

Team leadership

Operational management

Communication

AI evaluation

Vendor management

Outils

Python

Description du poste

White Circle is seeking a Data Labeling Lead to build and guide a scalable labeling program within a safety-focused AI environment. You will hire and coach a distributed team, set guidelines, and drive QA, calibration, and auditing processes in partnership with AI researchers and engineers.

You will manage vendor relationships, monitor quality and cost, and push for tooling enhancements. This role requires strong people skills and a startup mindset, with hybrid work options from Paris or London.