Remote Audio Quality Auditor (US Contractor)

AI Trainer Jobs; United States; Remote; USD 58,000 - 80,000; Part time

Job summary

AI Trainer Jobs is seeking an English (US) Audio Evaluation Specialist for a remote contractor role in the United States. You will review audio prompts and responses against a quality rubric, compare outputs, and provide structured feedback for retraining.

Responsibilities include tagging errors, maintaining calibration with gold standards, and ensuring consistent rubric application across batches. Flexible, remote work with approximately 10 hours per week.

Qualifications

• Experience evaluating audio annotation or similar content.
• Ability to apply multi-page rubrics consistently.
• Clear written reasoning naming issues and rubric clauses.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate audio annotation outputs against a versioned rubric.
• Compare paired responses and provide a rationale for the stronger answer.
• Label hallucinations, policy violations, and other errors with tags.
• Capture ambiguous prompts and route for rubric updates.
• Maintain reviewer scores calibrated against gold standards weekly.

Skills

Audio evaluation

Annotation experience

Linguistics background

Inter-rater calibration

Job description

AI Trainer Jobs is seeking an English (US) Audio Evaluation Specialist for a remote contractor role in the United States. You will review audio prompts and responses against a quality rubric, compare outputs, and provide structured feedback for retraining.

Responsibilities include tagging errors, maintaining calibration with gold standards, and ensuring consistent rubric application across batches. Flexible, remote work with approximately 10 hours per week.