Remote AI Data Evaluator: Video Review & Rubric Feedback

AuraOne, Inc.; United States; Remote; USD 34,000 - 55,000; Part time

Job summary

AuraOne, Inc. is seeking a diligent remote contractor to evaluate at-work video recorder outputs against a versioned rubric. You will assign severity tags, write rationales, and help identify edge cases for quality control.

Responsibilities include comparing responses, labeling issues, calibrating against gold standards, and documenting recurring failure modes to guide future training iterations.

Qualifications

• Prior evaluation, annotation, or human-rater experience on At-Work Video Recorder [for Business Owners] evaluation or adjacent content.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with policy tags.
• Capture ambiguous prompts and escalate for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples weekly.
• Document recurring failure modes for model improvement.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

At Work Video Recorder for Business</n

Design

Job description

AuraOne, Inc. is seeking a diligent remote contractor to evaluate at-work video recorder outputs against a versioned rubric. You will assign severity tags, write rationales, and help identify edge cases for quality control.

Responsibilities include comparing responses, labeling issues, calibrating against gold standards, and documenting recurring failure modes to guide future training iterations.