Remote English Voice Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 55,000 - 96,000; Full time

Benefits offered by this job

Remote work

Job summary

AuraOne is seeking a remote contractor to serve as an English Voice Coach reviewer. You will assess model outputs against a quality rubric, label edge cases, and provide structured feedback for retraining.

Responsibilities include comparing paired responses, tagging unsafe content, and calibrating reviewer scores weekly. The role requires strong attention to detail, prior evaluation or annotation experience, and the ability to apply rubrics consistently across large batches.

Qualifications

• Prior evaluation, annotation, or human-rater experience for English Voice Coach work.
• Ability to apply rubrics consistently across long batches.
• Clear written reasoning naming the issue and the rubric clause being applied.
• Strong attention to detail and flagging when a prompt is the problem.

Responsibilities

• Evaluate english voice coach evaluation model outputs against a versioned rubric and assign severity tags for English Voice Coach assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Evaluation experience

Rubric application

Written reasoning

Attention to detail

Weekly availability

Linguistics

Content moderation

Trust & safety

Inter-rater metrics

Calibration cycles

Job description

AuraOne is seeking a remote contractor to serve as an English Voice Coach reviewer. You will assess model outputs against a quality rubric, label edge cases, and provide structured feedback for retraining.

Responsibilities include comparing paired responses, tagging unsafe content, and calibrating reviewer scores weekly. The role requires strong attention to detail, prior evaluation or annotation experience, and the ability to apply rubrics consistently across large batches.