Remote Audio Evaluation & Feedback Specialist

AI Trainer Jobs; United States; Remote; USD 57,000 - 81,000; Full time

Job summary

AuraOne is seeking a Meeting Audio Conversation Evaluator for a remote contractor role. You will review audio annotation prompts and responses, label edge cases, and provide structured feedback to help retrain the model.

Evaluate outputs against a quality rubric and document your findings with clear rationale. Responsibilities include comparing outputs, tagging issues, and calibrating against gold standards weekly.

Qualifications

• Experience in audio annotation or human data review.
• Ability to apply multi-page rubrics consistently.
• Strong written reasoning and clear issue naming.
• Reliable 10+ hours per week availability.

Responsibilities

• Evaluate audio annotation model outputs against a rubric and tag severity.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and suggest rubric updates to the team.
• Maintain reviewer-quality scores by calibrating against gold-standard examples.

Skills

Audio annotation experience

Attention to detail

Written reasoning

Remote/async work

Job description

AuraOne is seeking a Meeting Audio Conversation Evaluator for a remote contractor role. You will review audio annotation prompts and responses, label edge cases, and provide structured feedback to help retrain the model.

Evaluate outputs against a quality rubric and document your findings with clear rationale. Responsibilities include comparing outputs, tagging issues, and calibrating against gold standards weekly.