Remote AI Evaluation Specialist

YO AI Labs; Abu Dhabi; Remote; AED 220,000 - 331,000; Full time

Job summary

YO AI Labs is seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise training project. This is a remote contractor position open to candidates in the US, Canada, UK, Ireland, Australia, and New Zealand.

You will evaluate outputs against rubrics, provide actionable feedback, and help refine evaluation criteria. No formal AI experience required; strong writing and analytical skills are essential for the role.

Qualifications

• Experience in grading, quality assurance, or editorial review.
• Daily use of AI assistants like ChatGPT or Claude.
• Ability to analyze complex information and write clearly.
• Experience with rubric development or quality assessment.
• Attention to detail and consistency in evaluation.

Responsibilities

• Evaluate AI outputs against rubrics for accuracy and relevance.
• Identify errors and provide actionable feedback.
• Maintain documentation of evaluations and recommendations.
• Contribute to rubric interpretation discussions.
• Support process improvements and best practices.

Skills

Quality assurance

Editorial review

Annotation

Analytical thinking

Rubric development

Process improvement

Independent work

Tools

ChatGPT

Claude

Job description

YO AI Labs is seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise training project. This is a remote contractor position open to candidates in the US, Canada, UK, Ireland, Australia, and New Zealand.

You will evaluate outputs against rubrics, provide actionable feedback, and help refine evaluation criteria. No formal AI experience required; strong writing and analytical skills are essential for the role.