Senior AI Evaluation & Analytics Scientist

Enlighten AI Labs; Ciudad de México; On-site; MXN 900,000 - 1,300,000; Full time

Job summary

Enlighten AI Labs is seeking a Senior Data Scientist for AI Evaluation & Analytics in Mexico City. You will define what constitutes high‑quality analysis produced by AI agents, and direct agents to generate test cases and verify results against independent evidence to measure product improvements.

You will build representative evaluation sets, design checks, and trace failures to underlying data or code, enabling the product team to act on reliable insights.

Qualifications

• Senior data science experience evaluating complex outputs and diagnosing why a workflow produced the wrong result.
• Strong Python and SQL skills and sound statistical judgment about sampling, uncertainty, and evaluation design.
• Practical experience with language models, AI agents, and their failure modes in analysis or software workflows.
• Ability to build custom evaluation apps and orchestrate multiple agents while maintaining independent source checks.

Responsibilities

• Define what good analysis looks like, including accuracy, source support, completeness, reproducibility, and usefulness to the person making a decision.
• Build representative evaluation sets with reference answers and the source data needed to verify them. Document what each set can measure.
• Design deterministic checks, statistical comparisons, and expert reviews that provide independent evidence alongside model-based grading.
• Trace failures to data selection, calculations, reasoning, generated code, or presentation so the team can fix the underlying issue.
• Run multiple AI agents in parallel every day to generate test cases, write regression checks, and build custom tools for comparing runs and investigating failures.

Skills

Python

SQL

Statistics

Experiment design

Independent evaluation

Communication

Education

Master's degree in a quantitative field

Tools

Jupyter

Git

Job description

Enlighten AI Labs is seeking a Senior Data Scientist for AI Evaluation & Analytics in Mexico City. You will define what constitutes high‑quality analysis produced by AI agents, and direct agents to generate test cases and verify results against independent evidence to measure product improvements.

You will build representative evaluation sets, design checks, and trace failures to underlying data or code, enabling the product team to act on reliable insights.