Remote AI Quality Evaluator & Feedback Specialist

24-Mag Llc; Northern (KY); Hybrid; USD 41,000 - 124,000; Part time

Job summary

24-MAG LLC offers a specialised part-time consulting opportunity for experienced professionals to contribute to AI output evaluation and quality assessment. You will assess AI-generated content against rubrics, provide actionable feedback, and document findings to support model improvement.

The role is fully remote and contract-based. Ideal candidates have experience in evaluation, QA, editorial review, assessment, or annotation, and can work independently with high attention to detail.

Qualifications

• Experience in grading and quality assessment with meticulous feedback.
• Ability to write clear, actionable feedback and document assessments.
• Background in evaluation, QA, editorial review, or annotated analysis.

Responsibilities

• Evaluate AI outputs against rubrics and quality standards.
• Assess accuracy, relevance, completeness, and reasoning in responses.
• Provide concise written feedback and maintain evaluation records.
• Interpret rubrics, discuss ambiguous cases, and suggest improvements.
• Collaborate with reviewers to ensure consistency in workflows.

Skills

Quality assurance

Editorial review

Assessment & annotation

Written feedback

AI tools usage

Tools

ChatGPT

Claude

Job description

24-MAG LLC offers a specialised part-time consulting opportunity for experienced professionals to contribute to AI output evaluation and quality assessment. You will assess AI-generated content against rubrics, provide actionable feedback, and document findings to support model improvement.

The role is fully remote and contract-based. Ideal candidates have experience in evaluation, QA, editorial review, assessment, or annotation, and can work independently with high attention to detail.