Evaluation Engineer, Applied AI – Enterprise Impact

Mistral; Paris; Sur place; EUR 70 000 - 110 000; Plein temps

Avantages offerts par ce poste

PTO 218 days

Health insurance

Mobility allowance €600/year

Swile meal vouchers

Gymlib sponsorship

Parental policy: 4 extra weeks

Résumé du poste

Mistral is seeking an Evaluation Engineer in Paris to design and implement evaluation systems for enterprise AI models. You will work at the crossroads of research, engineering, and customer-facing teams, measuring model performance across diverse use cases and contributing to production-ready evaluation frameworks.

You will design scalable pipelines, develop novel methodologies, and collaborate with research and product teams to drive model improvements based on real-world customer needs.

Qualifications

• 3+ years of ML evaluation and benchmarking for LLMs.
• Strong experience in AI or machine learning product implementation with APIs/back-end.
• Excellent communication skills for technical and non-technical audiences.
• Fluent in English and able to explain complex concepts clearly.

Responsabilités

• Design and implement comprehensive evaluation frameworks to measure LLM capabilities across diverse use cases.
• Build scalable evaluation infrastructure and pipelines for rapid, reproducible assessments.
• Develop novel evaluation methodologies for emerging capabilities and domain-specific applications.
• Create custom evaluation suites tailored to enterprise needs in collaboration with customers.
• Collaborate with research teams to translate insights into model improvements and training decisions.
• Partner with product teams to improve evaluation tooling based on feedback.

Connaissances

ML evaluation

Benchmarking

Python

Communication

English fluency

Outils

PyTorch

HuggingFace Transformers

Description du poste

Mistral is seeking an Evaluation Engineer in Paris to design and implement evaluation systems for enterprise AI models. You will work at the crossroads of research, engineering, and customer-facing teams, measuring model performance across diverse use cases and contributing to production-ready evaluation frameworks.

You will design scalable pipelines, develop novel methodologies, and collaborate with research and product teams to drive model improvements based on real-world customer needs.