Remote Video Evaluation Specialist

AuraOne; United States; On-site; USD 21,000 - 28,000; Full time

Job summary

AuraOne is seeking a Video Evaluation Generalist contractor to review model outputs against our quality rubric, compare paired responses, and provide structured feedback to retrain the model. Remote and US-eligible, the role emphasizes attention to detail and clear, written reasoning.

You will label edge cases, annotate for hallucinations, and document recurring failure modes to support the modeling team. Requires 10+ hours/week availability and strong rubric discipline.

Qualifications

• Prior evaluation, annotation, or human-rater experience on video evaluation generalist evaluation or adjacent content.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against the versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them for rubric updates.
• Document recurring failure modes to guide retraining.

Skills

Attention to detail

Clear written reasoning

Async availability

Rubric-based evaluation

Video evaluation generalist experience

Job description

AuraOne is seeking a Video Evaluation Generalist contractor to review model outputs against our quality rubric, compare paired responses, and provide structured feedback to retrain the model. Remote and US-eligible, the role emphasizes attention to detail and clear, written reasoning.

You will label edge cases, annotate for hallucinations, and document recurring failure modes to support the modeling team. Requires 10+ hours/week availability and strong rubric discipline.