Remote TTS Conversation Evaluator – Quality Feedback

AI Trainer Jobs; United States; Remote; USD 57,000 - 81,000; Part time

Job summary

AI Trainer Jobs is seeking a Text-to-Speech Conversation Evaluator for a remote contractor position. You will review prompts and responses against AuraOne's rubric, label issues, and provide structured feedback to help retrain models. The role emphasizes careful reasoning, edge-case labeling, and consistent rubric application.

You will evaluate outputs, compare responses, and document findings. This flexible, part-time schedule supports at least 10 hours per week with US eligibility for remote

Qualifications

• Prior evaluation, annotation, or human-rater experience on text to speech conversation evaluation or adjacent content.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate text to speech conversation evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.

Skills

Evaluation experience

Attention to detail

Written reasoning

Async availability

Rubric-based annotation

Job description

AI Trainer Jobs is seeking a Text-to-Speech Conversation Evaluator for a remote contractor position. You will review prompts and responses against AuraOne's rubric, label issues, and provide structured feedback to help retrain models. The role emphasizes careful reasoning, edge-case labeling, and consistent rubric application.

You will evaluate outputs, compare responses, and document findings. This flexible, part-time schedule supports at least 10 hours per week with US eligibility for remote