AI Quality Evaluator (Remote Contractor)

YO AI Labs; Wrocław; Remote; PLN 160,000 - 321,000; Part time

Job summary

YO AI Labs is seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise training project. You will evaluate AI-generated responses against detailed rubrics, identify issues, and provide clear feedback.

No prior formal AI experience required. You will work remotely with teams across regions, handle high-volume evaluations, and help evolve quality standards through collaboration.

Qualifications

• Experience in grading, QA, editorial review, assessment, or similar analytical work.
• Advanced daily use of AI assistants like ChatGPT or Claude.
• Strong written communication to convey findings clearly.
• Ability to develop rubric criteria and improve evaluation processes.

Responsibilities

• Evaluate AI-generated outputs against rubrics for accuracy, relevance, quality.
• Identify reasoning gaps and inconsistencies across large datasets.
• Provide concise, actionable feedback for model improvements.
• Document evaluations and recommendations clearly and consistently.
• Participate in rubric interpretation discussions and standards evolution.

Skills

Quality assurance

Editorial review

Analytical thinking

Rubric development

Independent work

Written communication

Tools

ChatGPT

Claude

Job description

YO AI Labs is seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise training project. You will evaluate AI-generated responses against detailed rubrics, identify issues, and provide clear feedback.

No prior formal AI experience required. You will work remotely with teams across regions, handle high-volume evaluations, and help evolve quality standards through collaboration.