Head of Data Labeling & AI Evaluation

White Circle

Paris

Hybrid

EUR 90,000 - 130,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Relocation package
Hybrid work Paris/London
Equity
Paid time off

Job summary

White Circle in Paris invites a Data Labeling Lead to build from scratch and oversee a distributed team driving annotation quality and scalable workflows.

You will partner with AI researchers to define guidelines, monitor metrics, and manage vendor relationships; relocation package is available for Paris, or options to work from London. This hybrid role offers growth and equity opportunities.

Qualifications

  • Experience leading data annotation or AI evaluation teams.
  • Strong operational and people management skills.
  • Understanding of AI model evaluation, LLM behavior, and annotation workflows.
  • Ability to design scalable processes without sacrificing quality.
  • Excellent cross-team communication.

Responsibilities

  • Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management).
  • Define annotation guidelines, quality standards, and evaluation frameworks.
  • Develop QA processes, calibration sessions, and auditing systems.
  • Partner with AI researchers and engineers to translate research objectives into labeling workflows.
  • Prioritize labeling projects based on business and research needs.
  • Monitor operational metrics including quality, throughput and cost.
  • Improve annotation tooling and workflow efficiency.
  • Lead complex AI evaluation projects including safety, RLHF, policy evaluation, and benchmarks.
  • Analyse disagreement patterns and edge cases to improve guidelines and model performance.
  • Manage vendor relationships and ensure consistent quality across distributed teams.
  • Build reporting dashboards and communicate operational insights to leadership.
  • Foster a culture of continuous improvement, accountability, and operational excellence.

Skills

Leading data annotation teams
Operational management
People management
AI evaluation understanding
Communication across teams

Tools

SQL
Python

Job description

White Circle in Paris invites a Data Labeling Lead to build from scratch and oversee a distributed team driving annotation quality and scalable workflows.

You will partner with AI researchers to define guidelines, monitor metrics, and manage vendor relationships; relocation package is available for Paris, or options to work from London. This hybrid role offers growth and equity opportunities.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Labeling Lead for AI Safety & RLHF
Data Labeling Lead for AI Safety & RLHF

White Circle • Paris

Hybrid
EUR 70,000 - 110,000
Equity
Flexible time off
Office in Paris (2nd arrondissement)
+8
Head of Data Labeling
Head of Data Labeling

White Circle • Paris

Hybrid
EUR 90,000 - 130,000
Relocation package
Hybrid work Paris/London
Equity
+1
Senior Data Labeler
Senior Data Labeler

White Circle • Paris

Hybrid
EUR 70,000 - 110,000
Equity
Flexible time off
Office in Paris (2nd arrondissement)
+8
Senior Data Labeler
Senior Data Labeler

White Circle • Paris

Hybrid
EUR 90,000 - 130,000
Equity compensation
Hybrid work Paris
Relocation package
+3
Senior Tech Lead - Data Pipelines & AI (Paris On-Site)
Senior Tech Lead - Data Pipelines & AI (Paris On-Site)

Arlequin AI • Paris

On-site
EUR 80,000 - 100,000
80-100k+€ + BSPCE
On-site central Paris
Data Labeling Specialist
Data Labeling Specialist

Alignerr Corp. • France

Remote
EUR 17,000 - 34,000
Senior AI & Data Delivery Engineer
Senior AI & Data Delivery Engineer

Singulier by OMMAX • Paris

On-site
EUR 120,000 - 180,000
Lead Data Engineer — AI-Driven Cloud Data Architect (GCP)
Lead Data Engineer — AI-Driven Cloud Data Architect (GCP)

Innovela • Paris

Hybrid
EUR 90,000 - 130,000
Hybrid work model
Remote work flexibility
International collaboration
Data Annotation Operations Lead — Remote-Friendly
Data Annotation Operations Lead — Remote-Friendly

Spore.Bio • Paris

Hybrid
EUR 65,000 - 90,000
Remote work equipment budget
Gymlib subscription
Premium health insurance (Alan in FR)
+3
Lead Architecte Data & AI — Architecte Principal & Leader Data
Lead Architecte Data & AI — Architecte Principal & Leader Data

Prodware France • Paris

On-site
EUR 90,000 - 140,000
Télétravail
Plan épargne entreprise
Plan retraite
+1