Music & Lyrics Evaluation Specialist (Remote)

AuraOne; United States; Remote; USD 18,000 - 23,000; Part time

Job summary

AuraOne is seeking a remote Music lyrics evaluation expert (English for India) to review model outputs, label issues, and provide structured feedback. You will compare paired responses, annotate edge cases, and help calibrate the review process for high-quality data labelling.

As an independent contractor, you’ll work with a flexible schedule on US-based programs, earning $13–$17 per hour and supporting ongoing rubric improvements and audits.

Qualifications

• Prior evaluation or annotation experience in music lyrics evaluation or adjacent content.
• Clear written reasoning that names the issue and rubric clause being applied.
• Comfort applying multi-page rubrics consistently across long batches.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate music lyrics evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and provide a rationale for the stronger answer.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples weekly.

Skills

Music lyrics evaluation

Inter-rater calibration

Rubric-based annotation

Model output evaluation

Clear written reasoning

Job description

AuraOne is seeking a remote Music lyrics evaluation expert (English for India) to review model outputs, label issues, and provide structured feedback. You will compare paired responses, annotate edge cases, and help calibrate the review process for high-quality data labelling.

As an independent contractor, you’ll work with a flexible schedule on US-based programs, earning $13–$17 per hour and supporting ongoing rubric improvements and audits.