Remote Short-Form Video Evaluator & Rubric Annotator

AuraOne Human Data

United States

Remote

USD 34,000 - 49,000

Part time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Remote work
Contractor role

Job summary

AuraOne Human Data is seeking a remote Short-Form Video Creator evaluation contractor. You will review model outputs against a versioned rubric, label issues, and help retrain our system. The role is contract-based with flexible hours and requires consistent, written rationale for rubric decisions.

Responsibilities include comparing outputs, tagging errors, and maintaining calibration against gold-standard examples each week. Prior experience in evaluation or moderation is preferred.

Qualifications

  • Experience evaluating short form video creator outputs using rubrics.
  • Ability to apply rubrics consistently across large batches.
  • Clear written reasoning naming issues and rubric clauses.
  • Attention to detail and ability to flag problematic prompts.
  • Async availability for at least 10 hours per week.

Responsibilities

  • Evaluate model outputs against a rubric and assign severity tags.
  • Compare paired responses and select the stronger one with rationale.
  • Label hallucinations and unsafe content with structured tags.
  • Capture ambiguous prompts and route to rubric updates.
  • Calibrate scores against gold-standard examples weekly.
  • Document recurring failure modes for retraining.

Skills

Model output evaluation
Rubric-based annotation
Severity tagging
Inter-rater calibration
Short Form Video Creator evaluation
Design

Job description

AuraOne Human Data is seeking a remote Short-Form Video Creator evaluation contractor. You will review model outputs against a versioned rubric, label issues, and help retrain our system. The role is contract-based with flexible hours and requires consistent, written rationale for rubric decisions.

Responsibilities include comparing outputs, tagging errors, and maintaining calibration against gold-standard examples each week. Prior experience in evaluation or moderation is preferred.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Video Data Annotation Evaluator & Rubric Specialist
Remote Video Data Annotation Evaluator & Rubric Specialist

Auraone • United States

Remote
USD 28,000 - 41,000
Remote Video Content Evaluator & Rubric Specialist
Remote Video Content Evaluator & Rubric Specialist

Auraone • United States

Remote
USD 21,000 - 34,000
Remote Video Evaluation Specialist — Contract
Remote Video Evaluation Specialist — Contract

AuraOne Human Data • United States

Remote
USD 28,000 - 41,000
Remote Video Editing Evaluation Specialist
Remote Video Editing Evaluation Specialist

Auraone • United States

Remote
USD 57,000 - 81,000
Remote work
US-eligible
Short-Form Video Creator
Short-Form Video Creator

AuraOne Human Data • United States

Remote
USD 34,000 - 49,000
Remote work
Contractor role
Remote AI Data Evaluator: Video Review & Rubric Feedback
Remote AI Data Evaluator: Video Review & Rubric Feedback

AuraOne Human Data • United States

Remote
USD 34,000 - 55,000
Remote Audio-Video Evaluator & Rubric Feedback Analyst
Remote Audio-Video Evaluator & Rubric Feedback Analyst

AuraOne Human Data • United States

Remote
USD 21,000 - 39,000
Remote work
Contractor status
Remote Image/Video Evaluation & Feedback Specialist
Remote Image/Video Evaluation & Feedback Specialist

AuraOne Human Data • United States

Remote
USD 21,000 - 34,000
Video Content Review Lead (Remote Contractor)
Video Content Review Lead (Remote Contractor)

Auraone • United States

Remote
USD 34,000 - 62,000
Contractor
Remote work
Hourly pay after interview
Remote Video Instruction Evaluation Specialist
Remote Video Instruction Evaluation Specialist

AuraOne Human Data • United States

Remote
USD 39,000 - 52,000