Remote Video Data Evaluator & Rubric Annotator

Auraone

United States

Remote

USD 28,000 - 41,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

AI Trainer Jobs is seeking a Remote Video Data Collector (Smartphone) to review and label model outputs against our quality rubric. This independent contractor role requires careful annotation, calibration against rubrics, and the ability to provide clear reasoning for each decision.

You'll work asynchronously for at least 10 hours per week, reviewing video data prompts and responses, tagging issues, and helping improve future model performance.

Qualifications

  • Prior evaluation, annotation, or human-rater experience on video data collector evaluation or adjacent content.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the issue and rubric clause being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Responsibilities

  • Evaluate video data collector evaluation model outputs against a versioned rubric and assign severity tags for Video Data Collector (Smartphone) assignments.
  • Compare paired responses and pick the stronger answer with a written rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with structured tags.
  • Capture ambiguous prompts and route them back to the program team for rubric updates.
  • Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Evaluation / annotation experience
Rubric-based annotation
Written reasoning
Attention to detail
Async availability

Job description

AI Trainer Jobs is seeking a Remote Video Data Collector (Smartphone) to review and label model outputs against our quality rubric. This independent contractor role requires careful annotation, calibration against rubrics, and the ability to provide clear reasoning for each decision.

You'll work asynchronously for at least 10 hours per week, reviewing video data prompts and responses, tagging issues, and helping improve future model performance.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Video Data Reviewer: AI Quality & Feedback
Remote Video Data Reviewer: AI Quality & Feedback

Auraone • United States

Remote
USD 17,000 - 28,000
Remote Video Content Evaluator & Rubric Specialist
Remote Video Content Evaluator & Rubric Specialist

Auraone • United States

Remote
USD 21,000 - 34,000
Remote Data Evaluator for Model Feedback & Rubrics
Remote Data Evaluator for Model Feedback & Rubrics

Auraone • United States

Remote
USD 21,000 - 34,000
Remote Video Data Annotation Evaluator & Rubric Specialist
Remote Video Data Annotation Evaluator & Rubric Specialist

Auraone • United States

Remote
USD 28,000 - 41,000
Remote AI Data Evaluator: Video Review & Rubric Feedback
Remote AI Data Evaluator: Video Review & Rubric Feedback

AuraOne, Inc. • United States

Remote
USD 34,000 - 55,000
Remote Video Annotation Quality Reviewer
Remote Video Annotation Quality Reviewer

Auraone • United States

Remote
USD 28,000 - 55,000
Video Data Collector (Smartphone)
Video Data Collector (Smartphone)

Auraone • United States

Remote
USD 28,000 - 41,000
Remote AI Data Evaluator: Video Recorder Evaluation
Remote AI Data Evaluator: Video Recorder Evaluation

AuraOne Human Data • United States

Remote
USD 28,000 - 41,000
Remote Video Data Annotator for AI Training
Remote Video Data Annotator for AI Training

Remotebridge • United States

Remote
USD 21,000 - 34,000
Remote Video Annotation Evaluator — AI Data Labeling
Remote Video Annotation Evaluator — AI Data Labeling

AuraOne Human Data • United States

Remote
USD 21,000 - 34,000