Remote Audio Quality Evaluator for AI Chats

Annotation Academy; United States; Remote; USD 30,000 - 36,000; Full time

Job summary

Welo Data is seeking detail-oriented Audio Raters to evaluate conversations between users and AI voice agents. You will listen to recorded interactions and assess the quality of the AI agent's responses across two dimensions: Content and Prosody.

The role is freelance and remote, requiring English language proficiency and about 10 hours per week. Start ASAP with a flexible schedule; compensation is $22.10 per hour. You will provide concise rationales for each rating to support the scores.

Qualifications

• Strong listening and attention-to-detail skills.
• Ability to give consistent, well-reasoned ratings.
• Clear, concise written communication.
• Reliable availability for ~10 hours/week.

Responsibilities

• Listen carefully to recorded conversations between a person and an AI voice agent.
• Rate each of the agent's turns on two independent 1–5 scales: Content and Prosody.
• Write a short, specific, and clear justification for each score given.

Skills

Listening

Attention to detail

Consistency

Written communication

Job description

Welo Data is seeking detail-oriented Audio Raters to evaluate conversations between users and AI voice agents. You will listen to recorded interactions and assess the quality of the AI agent's responses across two dimensions: Content and Prosody.

The role is freelance and remote, requiring English language proficiency and about 10 hours per week. Start ASAP with a flexible schedule; compensation is $22.10 per hour. You will provide concise rationales for each rating to support the scores.