Research Engineer (Evals)

whitecircle

France

Hybride

EUR 70 000 - 95 000

Plein temps

14 jours+

Obtenez une réponse de cet employeur

Votre CV et votre lettre de motivation, adaptés exactement à ce que l’employeur recherche.

Passez les filtres ATS

Résumé du poste

White Circle is hiring a Research Engineer to build and maintain an internal benchmark suite for single and multi-turn content guardrails and agentic safety. You will work with the team on projects studying agent behaviours in the wild and help shape how AI safety is implemented in production.

You will write Python, build efficient LLM inference setups, and reproduce published benchmark results while extending evals to new features and data. Hybrid work from Paris or London may be supported.

Qualifications

  • Experience building an LLM benchmark that distinguishes model capabilities.
  • Experience creating synthetic data for post-training text or multimodal models.
  • Ability to reproduce published benchmark results and critique methodologies.
  • Proficiency in Python and writing production-ready code that can be extended by others.
  • Experience building efficient LLM inference solutions with orchestrated parallel calls and retry/rate-limit handling.

Responsabilités

  • Own and maintain the internal benchmark suite for single/multi-turn content guardrails and agentic safety.
  • Build benchmarks that differentiate model capabilities and support product evals.
  • Collaborate with product team to evaluate core functionality of flagship models.
  • Extend evals to new features and verticals, adapting to changing data.
  • Study realistic agentive and LLM failure modes and quantify them in the wild.

Connaissances

Python
LLM Benchmarking
Production Code
Research & Analysis

Outils

PyTorch
LangChain
Airflow

Description du poste

TLDR: We're looking for a research engineer to build and maintain our internal suite of benchmarks, covering single/multi-turn content and agentic guardrails. The research engineer would also work with the team on projects studying agent behaviours in the wild.

About us

White Circle is an AI Safety company building the safety, reliability, and optimization layer for AI systems. At the core of our platform are policies - simple natural-language rules that define what an AI model should and shouldn't do. We automatically test, enforce, and continuously improve these policies at scale.

We've raised $11M from top funds, founders, and senior leaders at OpenAI, Anthropic, HuggingFace, Mistral, DeepMind, Datadog, Sentry, and others

We process over 100M+ API calls every month

We fine-tune and train our own LLMs so they run faster and cheaper than any open or proprietary model

We're a small, highly focused team. If you want to work deeply on hard problems, see your work ship to production quickly, and influence how AI safety is actually built - you're the one we need.

About the team

White Circle's fundamental research team works on the science of how AI systems fail in the real world: where agents break, how misalignment actually looks like to the end user, and how it is present in the model internals. We build the evals, benchmarks, environments, and tooling that empirically study high-impact agent reliability concerns - some of which become the guardrails shipped in our products, and some of which become public writeups.

You will:
  • Own and maintain our internal benchmark suite, covering single/multi-turn content guardrails and agentic safety.
  • Build benchmarks that distinguish specific model capabilities.
  • Work with the product team to build evals covering core functionality of our flagship models.
  • Build benchmarks for new features coming out of the research team.
  • Adapt and extend evals to new verticals and changing product data.
  • Work on research projects that study and quantify realistic agentic and LLM failure modes in the wild.
You'll fit right in if you:
  • Have built an LLM benchmark from scratch that distinguished specific model capabilities (i.e., produced a measurable, defensible capability difference, not just a score).
  • Have built synthetic data for post-training textual or multimodal models.
  • Can reproduce a published benchmark result and identify where the original methodology is fragile or misleading.
  • You write Python that other people can build on. Our whole stack is Python; we want someone who has shipped and maintained production code and who factors messy problems into clean abstractions others can extend.
  • You can write efficient LLM inference setups, including sensible orchestration of parallel calls, retries, rate-limit handling.
  • An AI power-user - fluent with frontier models and coding agents day to day.

A big plus:

  • Automated red-teaming experience
  • Have worked across a range of agentic scaffolds and reproduced public benchmark results on them
  • Strong knowledge of existing reward-model / monitoring / safety benchmarks
  • One or more published papers in the evals / safety-evaluation space
Why White Circle

Paid time off in line with your local regulations, no matter where you work from.

Work from Paris (hybrid) with a relocation package available, or work from London ( note: we are unable to provide relocation support or private medical insurance for London-based roles for now ).

Comprehensive medical insurance for our France-based team

All the hardware, tools, and services you need

Covered subscriptions for AI agents and IDEs

Team off-sites twice a year: we've recently been to the Alps and to Saint-Tropez

How we hire
  • Introductory call with HR (25 min)
  • Take-home test task
  • Technical interview with Head of Fundamental Research (60 min)
  • Final conversation with our CEO (45 min)
Obtenez votre examen gratuit et confidentiel de votre CV.

ou faites glisser et déposez votre fichier ici.

Similar jobs

Postes similaires à comparer

Research Scientist/Engineer (Agentic Systems)
Research Scientist/Engineer (Agentic Systems)

whitecircle • France

Hybride
EUR 90 000 - 130 000
Paid time off
Hybrid Paris relocation package
Comprehensive medical insurance (FR)
+3
Head of Data Labeling
Head of Data Labeling

whitecircle • France

Hybride
EUR 80 000 - 110 000
Competitive salary + equity
Work from Paris (hybrid) with reloc.包e
Paid time off per local regulations
+3
Senior Data Labeler
Senior Data Labeler

White Circle • Paris

Hybride
EUR 90 000 - 130 000
Equity compensation
Hybrid work Paris
Relocation package
+3
Multimodal ML Engineer
Multimodal ML Engineer

whitecircle • France

Hybride
EUR 90 000 - 140 000
Hybrid work Paris with relocation
Relocation package available
Comprehensive health insurance
+2
Recruiter
Recruiter

White Circle • France

À distance
EUR 60 000 - 90 000
Salary and equity package
Paid time off
Hardware provided
+2
Lead ML Engineer
Lead ML Engineer

Ooak Data • Paris

Sur place
EUR 90 000 - 120 000
Alan health insurance (mutuelle)
50% Navigo covered
Senior Data Labeler
Senior Data Labeler

White Circle • Paris

Hybride
EUR 70 000 - 110 000
Equity
Flexible time off
Office in Paris (2nd arrondissement)
+8
AI Safety Benchmark Engineer — Hybrid (Relocation)
AI Safety Benchmark Engineer — Hybrid (Relocation)

whitecircle • France

Hybride
EUR 70 000 - 95 000
Research Scientist: Frontier AI Risk & Evaluations
Research Scientist: Frontier AI Risk & Evaluations

SaferAI • Paris

Hybride
EUR 90 000 - 140 000
Health insurance
Retirement plans
50% home transportation
+2
Design Engineer
Design Engineer

White Circle • France

Sur place
EUR 90 000 - 130 000
Equity
Flexible time off
Health insurance
+5