Remote AI Quality Evaluator

YO AI Labs; Doha; Remote; QAR 218,000 - 328,000; Full time

Job summary

YO AI Labs seeks an AI Evaluation Specialist for a remote, contractor role focused on assessing AI outputs against detailed rubrics and improving overall quality. You will identify issues, provide actionable feedback, and help refine evaluation criteria using established guidelines.

The ideal candidate has experience in QA, editorial review, or similar analytical work and is proficient with AI assistants like ChatGPT or Claude.

Qualifications

• Experience in grading, quality assurance, editorial review, assessment, annotation, or similar analytical work.
• Advanced daily use of AI assistants such as ChatGPT, Claude, or similar tools.
• Strong ability to analyze complex information and communicate findings clearly in writing.

Responsibilities

• Evaluate AI-generated outputs against rubrics for accuracy, relevance, quality, and guidelines.
• Apply consistent, objective judgments across large volumes of examples.
• Identify reasoning gaps, tool-use issues, logical errors, and inconsistencies.
• Provide concise, actionable feedback highlighting strengths and areas for improvement.
• Participate in rubric interpretation discussions and evolving quality standards.
• Contribute to evaluation best practices and process improvements.
• Maintain accurate documentation of evaluations and recommendations.

Skills

Quality assessment

Editorial review

Analytical thinking

Written communication

Independent work

Tools

ChatGPT

Claude

Job description

YO AI Labs seeks an AI Evaluation Specialist for a remote, contractor role focused on assessing AI outputs against detailed rubrics and improving overall quality. You will identify issues, provide actionable feedback, and help refine evaluation criteria using established guidelines.

The ideal candidate has experience in QA, editorial review, or similar analytical work and is proficient with AI assistants like ChatGPT or Claude.