Senior Data Labeler

White Circle

Paris

Hybrid

EUR 70,000 - 110,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity
Flexible time off
Office in Paris (2nd arrondissement)
Relocation support
Private health insurance
Mental health support
Language lessons
Lunch and dinner provided
Learning & development
Tools & equipment
Team off-sites

Job summary

White Circle in Paris is seeking a Data Labeling Lead to build our annotation team from the ground up, setting quality standards, evaluation frameworks, and the labeling operations behind our safety evals, RLHF, and benchmarks.

You will own quality, throughput, and cost while collaborating with AI researchers to translate research goals into scalable labeling workflows and to manage vendor relationships and dashboards for leadership.

Qualifications

  • Experience leading data annotation or AI evaluation teams.
  • Strong operational and people management skills.
  • Understanding of AI model evaluation, LLM behavior, and annotation workflows.
  • Ability to design scalable processes without sacrificing quality.
  • Excellent communication across technical and non-technical teams.

Responsibilities

  • Build from scratch and lead the Data Labeling team (hiring, coaching, performance management).
  • Define annotation guidelines, quality standards, and evaluation frameworks.
  • Develop quality assurance processes, calibration sessions, and auditing systems.
  • Partner with AI researchers and engineers to translate research objectives into labeling workflows.
  • Prioritise labeling projects based on business and research needs.
  • Monitor operational metrics including quality, throughput, and cost.
  • Improve annotation tooling, automation, and workflow efficiency.
  • Lead complex AI evaluation projects, including safety, RLHF, and benchmarks.
  • Analyze disagreement patterns to improve guidelines and model performance.
  • Manage vendor relationships and ensure consistent quality across distributed teams.
  • Build reporting dashboards and communicate operational insights to leadership.
  • Foster a culture of continuous improvement and accountability.

Skills

Data labeling leadership
People management
AI evaluation
Annotation workflows
Process design
Clear communication

Job description

TL;DR: We're looking for a Data Labeling Lead to build our annotation team from zero – setting quality standards, designing evaluation frameworks, and running the labeling operations behind our safety evals, RLHF, and benchmarks. You'll own quality, throughput, and cost while working side by side with our AI researchers.

About us

White Circle is an AI Safety company building the safety, reliability, and optimization layer for AI systems. At the core of our platform are policies – simple natural-language rules that define what an AI model should and shouldn’t do. We automatically test, enforce, and continuously improve these policies at scale.

  • We’ve recently raised our Series A funding round, taking our total funding to $70M. Our investors include top funds, founders, and senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, DeepMind, Datadog, Sentry, and others

  • We process over 100M+ API calls every month

  • We fine-tune and train our own LLMs so they run faster and cheaper than any open or proprietary model

We’re at a multi-million dollar run rate already having signed customers like Lovable and multiple neobanks. You’ll be joining at the most exciting time - early enough that the equity can be life-changing, but at a point where demand has been proven.

In this role, you will
  • Build from scratch and lead the Data Labeling team (hiring, coaching, and performance management)

  • Define annotation guidelines, quality standards, and evaluation frameworks

  • Develop quality assurance processes, calibration sessions, and auditing systems

  • Partner with AI researchers and engineers to translate research objectives into labeling workflows

  • Prioritise labeling projects based on business and research needs

  • Monitor operational metrics including quality, consistency, throughput, and cost

  • Improve annotation tooling, automation, and workflow efficiency

  • Lead complex AI evaluation projects, including safety, preference ranking, RLHF, policy evaluation, and benchmark creation

  • Analyse disagreement patterns and edge cases to improve guidelines and model performance

  • Manage vendor relationships and ensure consistent quality across distributed teams

  • Build reporting dashboards and communicate operational insights to leadership

  • Foster a culture of continuous improvement, accountability, and operational excellence

We're looking for someone who
  • Has experience leading data annotation or AI evaluation teams

  • Has strong operational and people management skills

  • Understands AI model evaluation, LLM behaviour, and modern annotation workflows

  • Can design scalable processes without sacrificing quality

  • Communicates clearly across technical and non-technical teams

  • Thrives in fast-moving startup environments

You might be a great fit if you
  • Have managed annotation programs for LLMs, generative AI, or machine learning

  • Have experience with RLHF, preference data collection, safety evaluations, or benchmark creation

  • Have worked in Trust & Safety, AI Safety, Content Moderation, or ML Ops

  • Have managed distributed or global annotation teams

  • Have experience with vendor management and outsourcing operations

Bonus points
  • Familiarity with prompt engineering and AI safety policies

  • SQL, Python, or data analysis experience

  • Experience building internal annotation platforms or workflow automation

  • Background in linguistics, cognitive science, machine learning, or data operations

Important note

This role involves overseeing projects that may include offensive, harmful, violent, sexual, or otherwise disturbing content.
You'll be responsible for ensuring reviewers have the tools, guidance, and support necessary to perform this work safely and consistently.

Compensation & benefits
  • Competitive compensation, including equity

  • Flexible time off

  • A spacious office in the heart of Paris’s 2nd arrondissement, with a flexible hybrid setup

  • Relocation support if you’re moving to Paris, available after your probationary period

  • Premium private health insurance

  • Mental health support, including coverage for therapy when you need it

  • Language lessons to help you improve your English or French

  • Lunch and dinner covered when you work from the office

  • Learning and development support for courses, conferences, and opportunities to grow your skills

  • All the hardware, subscriptions, tools, and services you need

  • Team off-sites twice a year: we’ve recently been to the Alps, Saint-Tropez, and Marbella

Process
  1. Intro call (25 min)

  2. Take-home test assignment

  3. Final conversation with CEO (45 min)

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Labeler
Senior Data Labeler

White Circle • Paris

Hybrid
EUR 90,000 - 130,000
Equity compensation
Hybrid work Paris
Relocation package
+3
Head of Data Labeling
Head of Data Labeling

whitecircle • France

Hybrid
EUR 80,000 - 110,000
Competitive salary + equity
Work from Paris (hybrid) with reloc.包e
Paid time off per local regulations
+3
Research Engineer (Evals)
Research Engineer (Evals)

whitecircle • France

Hybrid
EUR 70,000 - 95,000
Data Annotation Manager
Data Annotation Manager

Spore.Bio • Paris

Remote
EUR 55,000 - 75,000
Budget for remote work equipment
Gymlib subscription
Premium health insurance (Alan in FR)
+2
[Paris] Applied AI Architect, Industries
[Paris] Applied AI Architect, Industries

FDE Times • Paris

On-site
EUR 190,000 - 220,000
Senior Data Scientist
Senior Data Scientist

CybelAngel • Paris

Hybrid
EUR 60,000 - 70,000
Salary 60-70k EUR annually
5% bonus
Lunch card
+6
Lead Data Labeling & AI Evaluation
Lead Data Labeling & AI Evaluation

whitecircle • France

Hybrid
EUR 80,000 - 110,000
Competitive salary + equity
Work from Paris (hybrid) with reloc.包e
Paid time off per local regulations
+3
Lead ML Engineer
Lead ML Engineer

Ooak Data • Paris

On-site
EUR 90,000 - 120,000
Alan health insurance (mutuelle)
50% Navigo covered
[Paris] Applied AI Architect, Industries
[Paris] Applied AI Architect, Industries

Anthropic • Paris

Hybrid
EUR 130,000 - 180,000
AI/Data Engineering Manager (F/H)
AI/Data Engineering Manager (F/H)

Fabriq • Paris

Hybrid
EUR 90,000 - 100,000
7 weeks of paid leave per year
Alan health insurance
Swile meal card
+2