Remote AI Evaluation Specialist

YO AI Labs; Madrid; A distancia; EUR 43.000 - 73.000; Jornada completa

Descripción de la vacante

YO AI Labs seeks an AI Evaluation Specialist to assess and improve AI assistant outputs for an enterprise training project. This contract, remote role welcomes candidates in the US, Canada, UK, Ireland, Australia, and New Zealand.

You will evaluate responses against rubrics, provide actionable feedback, and help refine evaluation criteria. Strong writing, analytical thinking, and the ability to work independently are essential; experience in QA, editorial review, or annotation is preferred.

Formación

• Experience in grading, QA, editorial review, assessment, annotation or similar analytical work.
• Advanced, daily use of AI assistants such as ChatGPT, Claude or similar tools.
• Strong ability to analyze complex information and communicate findings clearly in writing.
• Experience with process improvement, rubric development, or quality assessment.
• Strong critical-thinking skills with a focus on consistency, accuracy, and fairness.

Responsabilidades

• Evaluate AI-generated outputs against rubrics for accuracy, relevance, quality, and guideline adherence.
• Apply consistent, objective judgment across large volumes of examples.
• Identify reasoning gaps, tool-use issues, and inconsistencies.
• Provide concise, actionable feedback highlighting strengths and areas for improvement.
• Participate in rubric interpretation discussions and evolving quality standards.
• Maintain accurate documentation of evaluations and recommendations.

Conocimientos

Quality evaluation

Analytical writing

Independent work

Communication

Consistency

Herramientas

ChatGPT

Claude

AI tools

Descripción del empleo

YO AI Labs seeks an AI Evaluation Specialist to assess and improve AI assistant outputs for an enterprise training project. This contract, remote role welcomes candidates in the US, Canada, UK, Ireland, Australia, and New Zealand.

You will evaluate responses against rubrics, provide actionable feedback, and help refine evaluation criteria. Strong writing, analytical thinking, and the ability to work independently are essential; experience in QA, editorial review, or annotation is preferred.