Research Engineer, AI Evaluation & Agent Reliability

Mollkom; Riyadh; Hybrid; SAR 180,000 - 360,000; Full time

Job summary

Mollkom in Riyadh is seeking a specialist in ML evaluation to design benchmarks for catalog, content, conversations and tool-use tasks, and to develop regression and adversarial tests.

You will analyze experiments, communicate findings clearly, and translate insights into engineering improvements. This hybrid role is based in Riyadh, Saudi Arabia.

Qualifications

• Experience in ML evaluation, applied research or AI quality engineering.
• Strong Python, data analysis and experiment design.
• Understanding of LLM-as-judge limitations, data leakage and evaluation bias.
• Ability to communicate results clearly and translate findings into engineering improvements.

Responsibilities

• Build benchmarks for catalog, content, conversations and tool-use tasks.
• Design rubrics and human/automated evaluation with agreement, bias and coverage analysis.
• Develop regression and adversarial tests for permissions and unsupported claims.
• Analyze experiments and model comparisons, communicating actionable findings and statistical limits.

Skills

Python

Data analysis

Experiment design

LLM evaluation

Job description

Commerce outcomes require more than fluent answers. Design evaluations that distinguish good writing, correct information and completed action, helping the team release improvements with measurable effects.

Responsibilities

• Build benchmarks for catalog, content, conversations and tool-use tasks.
• Design rubrics and human/automated evaluation with agreement, bias and coverage analysis.
• Develop regression and adversarial tests for permissions and unsupported claims.
• Analyze experiments and model comparisons, communicating actionable findings and statistical limits.

Qualifications

• Experience in ML evaluation, applied research or AI quality engineering.
• Strong Python, data analysis and experiment design.
• Understanding of LLM-as-judge limitations, data leakage and evaluation bias.
• Ability to communicate results clearly and translate findings into engineering improvements.

Hybrid in Riyadh, Saudi Arabia.