Remote Video Content Evaluator & Rubric Specialist

AI Trainer Jobs; United States; Remote; USD 21,000 - 34,000; Part time

Job summary

AI Trainer Jobs is seeking a Data Annotator - Video Content for a remote evaluation track to assess prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and provide structured feedback for retraining.

Responsibilities include evaluating model outputs against rubric, tagging content issues, maintaining calibration against gold standards, and routing ambiguous prompts to the program team for rubric updates.

Responsibilities

• Evaluate data annotator video content evaluation outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route to program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Job description

AI Trainer Jobs is seeking a Data Annotator - Video Content for a remote evaluation track to assess prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and provide structured feedback for retraining.

Responsibilities include evaluating model outputs against rubric, tagging content issues, maintaining calibration against gold standards, and routing ambiguous prompts to the program team for rubric updates.