Remote English Audio Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 58,000 - 80,000; Full time

Job summary

AI Trainer Jobs is seeking an English Audio Recording Expert (AUS) to work remotely reviewing audio prompts and responses against AuraOne's rubric. You will compare paired outputs, label edge cases, and provide structured feedback that the modeling team can use to retrain.

As a remote independent contractor, you will evaluate model outputs, annotate according to a versioned rubric, and participate in ongoing calibration to maintain high reviewer quality.

Qualifications

• Experience with audio annotation work for English prompts.
• Ability to apply rubrics consistently across long batches.
• Strong written reasoning naming the issue and rubric clause.

Responsibilities

• Evaluate audio annotation model outputs against a versioned rubric and assign severity tags for assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Audio annotation

Rubric-based evaluation

Inter-rater calibration

Quality review

Written reasoning

Job description

AI Trainer Jobs is seeking an English Audio Recording Expert (AUS) to work remotely reviewing audio prompts and responses against AuraOne's rubric. You will compare paired outputs, label edge cases, and provide structured feedback that the modeling team can use to retrain.

As a remote independent contractor, you will evaluate model outputs, annotate according to a versioned rubric, and participate in ongoing calibration to maintain high reviewer quality.