AI Safety Benchmark Engineer (Evals)

White Circle; Paris; Hybride; EUR 90 000 - 130 000; Plein temps

Avantages offerts par ce poste

Equity

Flexible time off

Hybrid office Paris/London

Relocation support

Health insurance

Mental health support

Office meals

Résumé du poste

White Circle, an AI Safety company, seeks a Research Engineer to own our internal benchmarks for single/multi-turn content guardrails and agent safety. You will build evals for flagship models and extend benchmarks as product data evolves, collaborating with the product team and research groups.

You will craft synthetic data, reproduce published benchmarks, and maintain production-grade code with clean abstractions. Office in Paris with hybrid setup; relocation support may apply.

Qualifications

• Built an LLM benchmark from scratch that distinguished model capabilities
• Generated synthetic data for post-training textual or multimodal models
• Could reproduce a published benchmark result and assess the methodology's robustness
• Proficient in Python with production-grade code and clean abstractions

Responsabilités

• Own and maintain internal benchmark suite for single/multi-turn content guardrails and agentic safety
• Build benchmarks that distinguish specific model capabilities
• Collaborate with product team to build evals for flagship models
• Extend evals to new features and data across verticals
• Study and quantify realistic agentic and LLM failure modes in the wild

Connaissances

Python

LLM Benchmarking

Production-ready code

Parallel inference

Agent safety

Frontier models

Research experience

Data synthesis

Outils

Git

Docker

Kubernetes

PyTorch

Description du poste

White Circle, an AI Safety company, seeks a Research Engineer to own our internal benchmarks for single/multi-turn content guardrails and agent safety. You will build evals for flagship models and extend benchmarks as product data evolves, collaborating with the product team and research groups.

You will craft synthetic data, reproduce published benchmarks, and maintain production-grade code with clean abstractions. Office in Paris with hybrid setup; relocation support may apply.