Freelance AI Project Reviewer — Shape Learner Feedback

MasterClass; United States; Remote; USD 103,000 - 172,000; Full time

Job summary

MasterClass is a freelance AI Project Reviewer role where you evaluate learner submissions against rubrics, providing pass/fail judgments and actionable feedback. You’ll calibrate assessment criteria with product and learning design teams to ensure fair outcomes for our members.

This temporary, flexible engagement is about 20 hours per week, with responsibilities including comparing human and AI assessments, documenting reasoning, and maintaining high evaluation standards.

Qualifications

• 5+ years of experience in assessment, evaluation, or educational feedback roles.
• Ability to apply rubrics consistently across many submissions.
• Strong written communication and feedback-writing skills.
• Comfort with independent work and meeting high evaluation standards.
• Interest in AI-driven evaluation methods.

Responsibilities

• Review learner submissions against detailed rubrics and provide consistent pass/fail judgments.
• Write clear, actionable feedback to guide learner improvement.
• Collaborate with learning design and product teams to refine rubrics and criteria.
• Compare human and AI assessment outcomes to calibrate scoring.
• Document reasoning behind evaluation decisions to support consistency.

Skills

Written communication

Detail-oriented

Independent worker

Feedback provision

AI evaluation curiosity

Job description

MasterClass is a freelance AI Project Reviewer role where you evaluate learner submissions against rubrics, providing pass/fail judgments and actionable feedback. You’ll calibrate assessment criteria with product and learning design teams to ensure fair outcomes for our members.

This temporary, flexible engagement is about 20 hours per week, with responsibilities including comparing human and AI assessments, documenting reasoning, and maintaining high evaluation standards.