AI Safety Benchmark Engineer

White Circle; Paris; Hybrid; EUR 70,000 - 110,000; Full time

Benefits offered by this job

Paid time off

Hybrid Paris relocation package

Comprehensive medical insurance (Paris

Comprehensive medical insurance for FR

All hardware and tools provided

Subscriptions for AI agents and IDEs

Team off-sites twice a year

Job summary

White Circle is hiring a research engineer to build and maintain an internal benchmark suite for single/multi-turn content guardrails and agentic safety. You will work with the fundamentals team to study agent behaviours in the wild and to develop evals for flagship models.

The role requires Python expertise, production-grade code, and experience creating benchmarks that reveal true model capabilities. Hybrid work from Paris, with relocation support considered.

Qualifications

• Have built an LLM benchmark from scratch that distinguished model capabilities.
• Have built synthetic data for post-training textual or multimodal models.
• Can reproduce a published benchmark result and identify fragile methodology.
• You write Python that other people can build on and maintain production code.

Responsibilities

• Own and maintain our internal benchmark suite for single/multi-turn content guardrails and agentic safety.
• Build benchmarks that distinguish specific model capabilities.
• Collaborate with the product team to build evals covering core functionality of flagship models.
• Develop benchmarks for new features from the research team.
• Adapt evals to new verticals and changing product data.
• Lead research projects studying realistic agentic and LLM failure modes in the wild.

Skills

Python

LLM benchmarks

Production code

Parallel calls

Agentic safety

Synthetic data

Job description

White Circle is hiring a research engineer to build and maintain an internal benchmark suite for single/multi-turn content guardrails and agentic safety. You will work with the fundamentals team to study agent behaviours in the wild and to develop evals for flagship models.

The role requires Python expertise, production-grade code, and experience creating benchmarks that reveal true model capabilities. Hybrid work from Paris, with relocation support considered.