Remote Music Evaluator & Rubric Auditor

AuraOne Human Data

United States

Remote

USD 28,000 - 55,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Remote work

Job summary

AuraOne Human Data is seeking a Music Annotator for a remote evaluation track. You will review music annotator prompts and responses, apply the quality rubric, and provide structured feedback to help retrain models.

Responsibilities include comparing outputs, labeling issues with the rubric, and maintaining calibration scores weekly. This contractor role emphasizes precise reasoning and attention to detail with flexible, asynchronous hours.

Qualifications

  • Prior evaluation/annotation or human-rater experience in music annotator evaluation.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the issue and the rubric clause being applied.
  • Strong attention to detail and ability to flag when a prompt is the problem.
  • Reliable async availability for at least 10 hours per week.

Responsibilities

  • Evaluate model outputs against rubric and assign severity tags.
  • Compare paired responses and select stronger with rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with tags.
  • Capture ambiguous prompts and route them to the program team for rubric updates.
  • Calibrate reviewer-quality scores against gold-standard examples weekly.
  • Document recurring failure modes for future training.

Skills

Model output evaluation
Rubric-based annotation
Severity tagging
Inter-rater calibration
Music Annotator evaluation
Sound and Audio
Humanities & Interdisciplinary Studies

Job description

AuraOne Human Data is seeking a Music Annotator for a remote evaluation track. You will review music annotator prompts and responses, apply the quality rubric, and provide structured feedback to help retrain models.

Responsibilities include comparing outputs, labeling issues with the rubric, and maintaining calibration scores weekly. This contractor role emphasizes precise reasoning and attention to detail with flexible, asynchronous hours.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Music Annotator
Music Annotator

AuraOne Human Data • United States

Remote
USD 28,000 - 55,000
Remote work
Remote Audio-Video Evaluator & Rubric Feedback Analyst
Remote Audio-Video Evaluator & Rubric Feedback Analyst

AuraOne Human Data • United States

Remote
USD 21,000 - 39,000
Remote work
Contractor status
Remote Music Production Evaluation Specialist
Remote Music Production Evaluation Specialist

Auraone • United States

Remote
USD 32,000 - 74,000
Audacity Audio QA Evaluator — Remote Contractor
Audacity Audio QA Evaluator — Remote Contractor

AuraOne Human Data • United States

Remote
USD 21,000 - 34,000
Remote Audio QA Evaluator & Rubric Tagger
Remote Audio QA Evaluator & Rubric Tagger

AuraOne Human Data • United States

Remote
USD 34,000 - 62,000
Remote Music Editor QA Specialist
Remote Music Editor QA Specialist

Auraone • United States

Remote
USD 70,000 - 117,000
Music Producer & Audio Post Specialist - AI Evaluation (US Remote)
Music Producer & Audio Post Specialist - AI Evaluation (US Remote)

AuraOne Human Data • United States

Remote
USD 96,000 - 165,000
Remote Music Audio Quality Reviewer - English (India)
Remote Music Audio Quality Reviewer - English (India)

Auraone • United States

Remote
USD 14,000 - 23,000
Music and Audio Experts
Music and Audio Experts

AuraOne Human Data • United States

Remote
USD 34,000 - 62,000
Music AI Quality Reviewer (Remote Contractor)
Music AI Quality Reviewer (Remote Contractor)

AuraOne Human Data • United States

Remote
USD 34,000 - 55,000