Remote Audio Evaluation Specialist, English (NZ)

AI Trainer Jobs; United States; Remote; USD 69,000 - 83,000; Full time

Job summary

AuraOne is seeking an English (New Zealand) Audio Evaluation Specialist to work remotely as an independent contractor. You will review audio annotation prompts and model outputs, apply the quality rubric, and write structured feedback to retrain the system.

Responsibilities include comparing paired responses, labeling edge cases, and documenting rationale. Strong attention to detail and reliable async availability are required.

Qualifications

• Experience evaluating audio annotations against a rubric.
• Ability to label edge cases and provide structured feedback.
• Familiarity with inter-rater calibration and audit tracking.

Responsibilities

• Evaluate audio annotation model outputs against a versioned rubric.
• Compare paired responses and provide rationale for the stronger answer.
• Label hallucinations, instruction-following failures, and unsafe content.
• Capture ambiguous prompts and route for rubric updates.
• Maintain reviewer-quality scores through weekly calibration.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Audio annotation

Job description

AuraOne is seeking an English (New Zealand) Audio Evaluation Specialist to work remotely as an independent contractor. You will review audio annotation prompts and model outputs, apply the quality rubric, and write structured feedback to retrain the system.

Responsibilities include comparing paired responses, labeling edge cases, and documenting rationale. Strong attention to detail and reliable async availability are required.