Remote Audio Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 55,000 - 83,000; Part time

Job summary

AuraOne is seeking an Audio Expert for a remote evaluation track. You will review audio annotation prompts and responses against AuraOne's quality rubric, label edge cases, and provide structured feedback to retrain models.

You will judge voice agents and speech recognition, including pronunciation and conversation quality. The role involves evaluating model outputs, tagging issues with a rubric-based system, and maintaining calibration against gold standards on a weekly basis.

Qualifications

• Prior evaluation, annotation, or human-rater experience on audio annotation or adjacent content for Audio Expert work.

Responsibilities

• Evaluate audio annotation model outputs against a versioned rubric and assign severity tags for Audio Expert assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Audio evaluation

Annotation review

Rubric tagging

Inter-rater calibration

Written reasoning

Attention to detail

Async availability

Job description

AuraOne is seeking an Audio Expert for a remote evaluation track. You will review audio annotation prompts and responses against AuraOne's quality rubric, label edge cases, and provide structured feedback to retrain models.

You will judge voice agents and speech recognition, including pronunciation and conversation quality. The role involves evaluating model outputs, tagging issues with a rubric-based system, and maintaining calibration against gold standards on a weekly basis.