Frontier AI Evaluation Engineer: Build Next-Gen Benchmarks

Mistral; Paris; On-site; EUR 70,000 - 110,000; Full time

Job summary

Mistral in Paris is seeking an Evaluation Engineer to own evals from start to finish, running workshops with customers and industry experts to identify the most impactful use cases. You will design benchmarks, source data, score outcomes, and run evaluations across new Mistral models and competitors.

You will collaborate with post-training teams to tighten gaps, increase challenge, and present findings to senior leaders for strategic decisions.

Qualifications

• Experience planning and running model evaluations with stakeholders.
• Ability to design benchmarks and score outcomes across multiple models.

Responsibilities

• Own evals from start to finish, including workshops with customers and industry experts.
• Collaborate with post-training teams and present findings to senior leaders for strategic decisions.

Job description

Mistral Ai is seeking an Evaluation Engineer to own evals from start to finish, running workshops with customers and industry experts to identify the most impactful use cases. You will design benchmarks, source data, score outcomes, and run evaluations across new Mistral models and competitors.

You will collaborate with post-training teams to tighten gaps, increase challenge, and present findings to senior leaders for strategic decisions.