Head of Data Labeling

whitecircle

France

Hybrid

EUR 80,000 - 110,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Competitive salary + equity
Work from Paris (hybrid) with reloc.包e
Paid time off per local regulations
All hardware, tools, and services you\
Subscriptions for AI agents and IDEs
Team off-sites twice a year

Job summary

White Circle is seeking a Lead for the Data Labeling team in a hybrid Paris-based role. You will hire, coach, and manage labeling staff, set guidelines, and build QA/auditing systems to ensure high-quality AI evaluation data.

You will collaborate with AI researchers and engineers to translate research goals into scalable labeling workflows, prioritize projects, and monitor metrics for quality, throughput, and cost. This role thrives in a fast-moving startup environment.

Qualifications

  • Experience leading data annotation or AI evaluation teams.
  • Strong operational and people management skills.
  • Understand AI model evaluation and annotation workflows.
  • Ability to design scalable processes without sacrificing quality.
  • Clear communication across technical and non-technical teams.
  • Thrives in fast-moving startup environments.

Responsibilities

  • Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management).
  • Define annotation guidelines, quality standards, and evaluation frameworks.
  • Develop QA processes, calibration sessions, and auditing systems.
  • Partner with AI researchers and engineers to translate research objectives into labeling workflows.
  • Prioritize labeling projects based on business and research needs.
  • Monitor metrics such as quality, throughput, and cost.
  • Improve annotation tooling, automation, and workflow efficiency.
  • Lead complex AI evaluation projects including safety, RLHF, and benchmarks.
  • Analyze disagreement patterns to improve guidelines and model performance.
  • Manage vendor relationships and ensure consistent quality across distributed teams.
  • Build reporting dashboards and communicate insights to leadership.
  • Foster a culture of continuous improvement and operational excellence.

Skills

Data labeling leadership
People management
AI evaluation knowledge
Process design
Cross-functional communication
Startup experience
Vendor management

Tools

Annotation tooling
Workflow automation

Job description

About us

White Circle is an AI Safety company building the safety, reliability, and optimization layer for AI systems. At the core of our platform are policies - simple natural-language rules that define what an AI model should and shouldn't do. We automatically test, enforce, and continuously improve these policies at scale.


We've raised $11M from top funds, founders, and senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, DeepMind, Datadog, Sentry, and others


We process over 100M+ API calls every month


We fine-tune and train our own LLMs so they run faster and cheaper than any open or proprietary model


We're at a multi-million dollar run rate already having signed customers like Lovable and multiple neobanks. You'll be joining at the most exciting time - early enough that the equity can be life changing but at a point where demand has been proven.


In this role, you will


  • Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management)

  • Define annotation guidelines, quality standards, and evaluation frameworks

  • Develop quality assurance processes, calibration sessions, and auditing systems

  • Partner with AI researchers and engineers to translate research objectives into labeling workflows

  • Prioritise labeling projects based on business and research needs

  • Monitor operational metrics including quality, consistency, throughput, and cost

  • Improve annotation tooling, automation, and workflow efficiency

  • Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation

  • Analyse disagreement patterns and edge cases to improve guidelines and model performance

  • Manage vendor relationships and ensure consistent quality across distributed teams

  • Build reporting dashboards and communicate operational insights to leadership

  • Foster a culture of continuous improvement, accountability, and operational excellence


We're looking for someone who


  • Has experience leading data annotation or AI evaluation teams

  • Has strong operational and people management skills

  • Understands AI model evaluation, LLM behavior, and modern annotation workflows

  • Can design scalable processes without sacrificing quality

  • Communicates clearly across technical and non-technical teams

  • Thrives in fast-moving startup environments


You might be a great fit if you


  • Have managed annotation programs for LLMs, generative AI, or machine learning

  • Have experience with RLHF, preference data collection, safety evaluations, or benchmark creation

  • Have worked in Trust & Safety, AI Safety, Content Moderation, or ML Ops

  • Have managed distributed or global annotation teams

  • Have experience with vendor management and outsourcing operations


Bonus points


  • Familiarity with prompt engineering and AI safety policies

  • SQL, Python, or data analysis experience

  • Experience building internal annotation platforms or workflow automation

  • Background in linguistics, cognitive science, machine learning, or data operations


Important note

This role involves overseeing projects that may include offensive, harmful, violent, sexual, or otherwise disturbing content. You'll be responsible for ensuring reviewers have the tools, guidance, and support necessary to perform this work safely and consistently.


Why White Circle


  • Competitive salary + equity

  • Work from Paris (hybrid) with a relocation package available, or work from London (note: we are currently unable to provide relocation support and medical insurance for London-based roles)

  • Paid time off in line with your local regulations

  • All the hardware, tools, and services you need

  • Covered subscriptions for AI agents and IDEs

  • Team off-sites twice a year: we've recently been to the Alps and Saint-Tropez

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Labeler
Senior Data Labeler

White Circle • Paris

Hybrid
EUR 90,000 - 130,000
Equity compensation
Hybrid work Paris
Relocation package
+3
Senior Data Labeler
Senior Data Labeler

White Circle • Paris

Hybrid
EUR 70,000 - 110,000
Equity
Flexible time off
Office in Paris (2nd arrondissement)
+8
Research Engineer (Evals)
Research Engineer (Evals)

whitecircle • France

Hybrid
EUR 70,000 - 95,000
Multimodal ML Engineer
Multimodal ML Engineer

whitecircle • France

Hybrid
EUR 90,000 - 140,000
Hybrid work Paris with relocation
Relocation package available
Comprehensive health insurance
+2
Research Scientist/Engineer (Agentic Systems)
Research Scientist/Engineer (Agentic Systems)

whitecircle • France

Hybrid
EUR 90,000 - 130,000
Paid time off
Hybrid Paris relocation package
Comprehensive medical insurance (FR)
+3
Finance Ops
Finance Ops

White Circle • France

On-site
EUR 85,000 - 135,000
Health insurance
Mental health support
401(k) retirement plan
+3
Recruiter
Recruiter

White Circle • France

Remote
EUR 60,000 - 90,000
Salary and equity package
Paid time off
Hardware provided
+2
Web Designer (Communication)
Web Designer (Communication)

White Circle • France

Remote
EUR 55,000 - 85,000
Equity package
Paid time off
All hardware & tools provided
+1
Senior Member of Technical Staff, Safety and Security for Agents
Senior Member of Technical Staff, Safety and Security for Agents

Cohere • Paris

Hybrid
EUR 60,000 - 90,000
Inclusive work culture
Cutting-edge AI research team
Weekly lunch stipend
+5
DevOps Engineer
DevOps Engineer

Empleora • Paris

Hybrid
EUR 69,000 - 129,000
Equity
Relocation assistance