Production AI Engineer: LLM Prompts & Evaluation

Mistral; Paris; Sur place; EUR 70 000 - 110 000; Plein temps

Avantages offerts par ce poste

Healthcare coverage

Relocation support

Retirement plans

Wellness programmes

Meal & transport allowances

Résumé du poste

Mistral in Paris seeks an AI/ML engineer focused on product-quality improvements across search, chat, documents, or audio features. You will own end‑to‑end evaluation, define what good looks like, measure it, run experiments, and ship reliable enhancements.

You’ll design prompts and orchestration, run A/B tests, monitor observability, and contribute to model releases with canary/shadow strategies and rollback criteria.

Qualifications

• 3–4 years of experience in ML engineering or AI‑driven software.
• Strong TypeScript or Python skills.
• Production experience with LLMs, prompts, and tool calls.
• Hands-on with evals and A/B testing and metrics design.
• Comfort implementing in production code and observability practices.

Responsabilités

• Design and run evaluations for your product area.
• Define and track metrics that matter.
• Own prompt and orchestration design.
• Run A/B tests and analyze results.
• Set up observability for LLM calls.
• Operate model releases with canary and rollback criteria.
• Improve core behaviours like routing and retrieval quality.
• Create templates and documentation for evals.
• Partner with Science to diagnose regressions and post‑mortems.

Connaissances

TypeScript

Python

Production LLM

Eval & AB testing

Observability

Product mindset

Communication

Description du poste

Mistral in Paris seeks an AI/ML engineer focused on product-quality improvements across search, chat, documents, or audio features. You will own end‑to‑end evaluation, define what good looks like, measure it, run experiments, and ship reliable enhancements.

You’ll design prompts and orchestration, run A/B tests, monitor observability, and contribute to model releases with canary/shadow strategies and rollback criteria.