Remote AI Quality Evaluator (Contractor)

YO AI Labs; Austin (TX); À distance; USD 34 000 - 69 000; Plein temps

Résumé du poste

YO AI Labs is seeking an AI Evaluation Specialist to remotely assess enterprise AI outputs against detailed rubrics, measuring accuracy, relevance and quality. You will provide concise, actionable feedback to drive improvements.

Responsibilities include identifying reasoning gaps, tool-use issues and inconsistencies, maintaining clear documentation, and contributing to rubric evolution. Prior QA or editorial experience is preferred; daily use of AI assistants like ChatGPT or Claude is a plus.

Qualifications

• Experience in grading, QA, editorial review, assessment, annotation, or similar analytical work.
• Advanced, daily use of AI assistants such as ChatGPT, Claude, or similar tools.
• Strong ability to analyze complex information and communicate findings clearly in writing.
• Experience with process improvement, rubric development, or quality assessment.
• Strong critical-thinking skills with a focus on consistency, accuracy, and fairness.

Responsabilités

• Evaluate AI-generated outputs against detailed rubrics for accuracy, relevance, quality, and adherence to guidelines.
• Apply consistent and objective judgment across large volumes of examples.
• Identify reasoning gaps, tool-use issues, logical errors, and inconsistencies.
• Provide concise, actionable feedback highlighting strengths and areas for improvement.
• Participate in discussions around rubric interpretation and evolving quality standards.
• Contribute to process improvements and evaluation best practices.
• Maintain accurate documentation of evaluations and recommendations.

Connaissances

Grading & QA

Analytical writing

Annotation

Process improvement

Rubric development

Independent work

Attention to detail

Outils

ChatGPT

Claude

Description du poste

YO AI Labs is seeking an AI Evaluation Specialist to remotely assess enterprise AI outputs against detailed rubrics, measuring accuracy, relevance and quality. You will provide concise, actionable feedback to drive improvements.

Responsibilities include identifying reasoning gaps, tool-use issues and inconsistencies, maintaining clear documentation, and contributing to rubric evolution. Prior QA or editorial experience is preferred; daily use of AI assistants like ChatGPT or Claude is a plus.