AI Output Evaluator — Remote Contractor

YO AI Labs; Medellín; A distancia; EUR 44.000 - 67.000; Jornada completa

Descripción de la vacante

YO AI Labs is seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project. You will evaluate AI-generated responses against rubrics and provide actionable feedback.

No prior formal AI experience is required, and you will work remotely with high-volume evaluation tasks, collaborating to refine evaluation criteria and processes.

Formación

• Experience in grading, quality assurance, editorial review, assessment, annotation, or similar analytical work.
• Ability to apply rubric-based judgments consistently across large volumes of examples.
• Strong written communication to clearly document findings and recommendations.

Responsabilidades

• Evaluate AI-generated outputs against defined rubrics for accuracy, relevance, quality and guideline adherence.
• Apply consistent and objective judgment across lots of examples.
• Identify reasoning gaps, tool-use issues, and logical inconsistencies.
• Provide concise, actionable feedback to improve AI systems.
• Participate in rubric interpretation discussions and quality standard updates.
• Maintain documentation of evaluations and recommendations.

Conocimientos

QA experience

Editorial review

Annotation

Assessment

AI tools

Herramientas

ChatGPT

Claude

Descripción del empleo

YO AI Labs is seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project. You will evaluate AI-generated responses against rubrics and provide actionable feedback.

No prior formal AI experience is required, and you will work remotely with high-volume evaluation tasks, collaborating to refine evaluation criteria and processes.