Remote Audio-Video Evaluator & Rubric Feedback Analyst

AuraOne Human Data

United States

Remote

USD 21,000 - 39,000

Part time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Remote work
Contractor status

Job summary

Responsibilities include comparing paired responses, tagging content, and calibrating scores weekly to maintain quality. Prior experience in audio annotation or related content is expected.

Qualifications

  • Experience evaluating audio annotation or related content for Audio Video Annotator work.
  • Ability to apply multi-page rubrics consistently across long batches.
  • Clear written reasoning naming the issue and the rubric clause being applied.
  • Reliable async availability for at least 10 hours per week.

Responsibilities

  • Evaluate audio annotation model outputs against a versioned rubric and assign severity tags.
  • Compare paired responses and select the stronger answer with a written rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with structured tags.
  • Capture ambiguous prompts and route them to the program team for rubric updates.
  • Maintain reviewer-quality scores by calibrating against gold-standard examples.
  • Document recurring failure modes for future model training.

Skills

Audio annotation
Rubric-based evaluation
Inter-rater tagging
Remote work
Attention to detail

Job description

Responsibilities include comparing paired responses, tagging content, and calibrating scores weekly to maintain quality. Prior experience in audio annotation or related content is expected.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Video Data Annotation Evaluator & Rubric Specialist
Remote Video Data Annotation Evaluator & Rubric Specialist

Auraone • United States

Remote
USD 28,000 - 41,000
Audio Video Annotator
Audio Video Annotator

AuraOne Human Data • United States

Remote
USD 21,000 - 39,000
Remote work
Contractor status
Remote Audio QA Evaluator & Rubric Tagger
Remote Audio QA Evaluator & Rubric Tagger

AuraOne Human Data • United States

Remote
USD 34,000 - 62,000
Remote Image/Video Evaluation & Feedback Specialist
Remote Image/Video Evaluation & Feedback Specialist

AuraOne Human Data • United States

Remote
USD 21,000 - 34,000
Remote Music Evaluator & Rubric Auditor
Remote Music Evaluator & Rubric Auditor

AuraOne Human Data • United States

Remote
USD 28,000 - 55,000
Remote work
Audacity Audio QA Evaluator — Remote Contractor
Audacity Audio QA Evaluator — Remote Contractor

AuraOne Human Data • United States

Remote
USD 21,000 - 34,000
Remote Short-Form Video Evaluator & Rubric Annotator
Remote Short-Form Video Evaluator & Rubric Annotator

AuraOne Human Data • United States

Remote
USD 34,000 - 49,000
Remote work
Contractor role
Audio Editing & Production Expert (Audacity)
Audio Editing & Production Expert (Audacity)

AuraOne, Inc. • United States

Remote
USD 21,000 - 34,000
Remote Video Content Evaluator & Rubric Specialist
Remote Video Content Evaluator & Rubric Specialist

Auraone • United States

Remote
USD 21,000 - 34,000
Video Annotation Quality Auditor (Remote)
Video Annotation Quality Auditor (Remote)

AuraOne Human Data • United States

Remote
USD 28,000 - 50,000