AI Quality Evaluator — Remote Contractor

YO AI Labs; Santa Cruz (CA); A distancia; USD 55.000 - 85.000; Jornada completa

Descripción de la vacante

YO AI Labs is seeking an AI Evaluation Specialist for a remote contractor role. You will assess AI outputs against detailed rubrics and provide actionable feedback to improve system quality.

You will work independently on high-volume evaluations, identify issues such as inaccuracies, biases, and tool-use gaps, and collaborate to refine evaluation criteria. Experience in QA, annotation, or editorial review is preferred; familiarity with ChatGPT or Claude helps interpret rubric results.

Formación

• Experience in grading, QA, editorial review, annotation, or similar analytical work.
• Advanced daily use of AI assistants such as ChatGPT or Claude.
• Strong written communication to convey findings clearly.
• Experience with rubric development and quality assessment.

Responsabilidades

• Evaluate AI-generated outputs against rubrics for accuracy, relevance, and quality.
• Apply consistent, objective judgments across large volumes of examples.
• Identify reasoning gaps, tool-use issues, and inconsistencies.
• Provide concise, actionable feedback to improve AI systems.
• Participate in rubric interpretation discussions and evolving standards.
• Maintain documentation of evaluations and recommendations.

Conocimientos

Quality Assurance

Editorial Review

Annotation

Rubric Development

Independent Work

Herramientas

ChatGPT

Claude

Descripción del empleo

YO AI Labs is seeking an AI Evaluation Specialist for a remote contractor role. You will assess AI outputs against detailed rubrics and provide actionable feedback to improve system quality.

You will work independently on high-volume evaluations, identify issues such as inaccuracies, biases, and tool-use gaps, and collaborate to refine evaluation criteria. Experience in QA, annotation, or editorial review is preferred; familiarity with ChatGPT or Claude helps interpret rubric results.