Remote AI Evaluation Lead - Quality & Feedback

YO IT Consulting; Dubai; On-site; AED 111,600 - 178,560; Full time

Job summary

YO IT Consulting in Dubai seeks an AI Trainer Lead to evaluate model responses and annotate datasets. You will apply rubrics, score reasoning quality, and provide precise, actionable feedback to improve AI training workflows.

The role requires strong analytical skills and meticulous judgment in a fully remote, contract-based setting. Ideal candidates have a Bachelor's degree and proven experience evaluating large content volumes with high accuracy and clear documentation.

Qualifications

• Bachelor's degree from a recognized institution.
• Experience evaluating or scoring substantial volumes of content against detailed criteria.
• Strong attention to detail and a commitment to accuracy.

Responsibilities

• Evaluate AI-generated responses using defined rubrics, scoring criteria, and project metrics.
• Assess answers for factual accuracy, relevance, coherence, and instruction compliance.
• Annotate, categorize, label, and review datasets used for AI model training and evaluation.
• Identify hallucinated facts, unsupported conclusions, contradictions, and bias.
• Provide clear written explanations supporting evaluation scores and ranking decisions.

Skills

Attention to detail

English communication

Critical reading

Analytical reasoning

Data annotation

Model evaluation

Remote collaboration

Documentation

Rubric interpretation

Quality assurance

Education

Bachelor's degree

Tools

Annotation tools

Evaluation platforms

Quality trackers

Job description

YO IT Consulting in Dubai seeks an AI Trainer Lead to evaluate model responses and annotate datasets. You will apply rubrics, score reasoning quality, and provide precise, actionable feedback to improve AI training workflows.

The role requires strong analytical skills and meticulous judgment in a fully remote, contract-based setting. Ideal candidates have a Bachelor's degree and proven experience evaluating large content volumes with high accuracy and clear documentation.