Remote Audio QA Reviewer — Speaker Diarization & Feedback

AuraOne; United States; On-site; USD 27,552 - 55,104; Part time

Job summary

AuraOne is seeking a Speaker Diarization Audio QA Reviewer for remote evaluation work. You will review audio annotation prompts and responses against AuraOne's quality rubric, compare outputs, and provide structured feedback for model retraining.

This independent contractor role requires clear written reasoning, meticulous attention to detail, and reliable async availability of at least 10 hours per week. Remote work is US-eligible, with hourly compensation to be confirmed after interview.

Qualifications

• Prior evaluation or annotation experience on audio content.
• Ability to apply multi-page rubrics across long batches.
• Clear written reasoning linking issues to rubric clauses.
• Strong attention to detail and ability to flag prompt problems.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate audio annotation model outputs against rubric and assign severity tags for Speaker Diarization Audio QA Reviewer assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
• Document recurring failure modes so the modeling team can target them in the next training run.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Audio annotation

Speech evaluation

Voice QA

Audio review

Speaker

Diarization

Job description

AuraOne is seeking a Speaker Diarization Audio QA Reviewer for remote evaluation work. You will review audio annotation prompts and responses against AuraOne's quality rubric, compare outputs, and provide structured feedback for model retraining.

This independent contractor role requires clear written reasoning, meticulous attention to detail, and reliable async availability of at least 10 hours per week. Remote work is US-eligible, with hourly compensation to be confirmed after interview.