Voice Quality Evaluator for AI Conversations

Innodata Inc.; United States; Remote; USD 28,000 - 37,000; Full time

Job summary

Innodata Inc. seeks detail-oriented Voice Specialists to evaluate AI models via real-time, voice-based conversations. You will roleplay with two models, compare them under consistent scenarios, and document observations across five evaluation dimensions.

The role emphasizes fair testing, recording, and producing a reasoned final choice with a clear rationale grounded in observed performance and guidelines.

Qualifications

• Bachelors Degree required and strong attention to detail.
• Excellent listening and comprehension skills.
• Clear written communication to explain observations.
• Ability to follow detailed evaluation guidelines objectively.

Responsibilities

• Review an assigned scenario and roleplay conversations with two AI models.
• Conduct comparable conversations with Model A and Model B with consistency.
• Maintain similar number of turns to support fair comparison.
• Record voice during interactions and assess responses for quality.
• Evaluate across five defined evaluation dimensions.
• Identify and categorize error clusters in audio or conversation.
• Compare performance and select the stronger model.
• Write a detailed rationale referencing specific observations.
• Apply evaluation guidelines consistently across scenarios.

Skills

Attention to detail

Listening skills

Written communication

Evaluation guidelines

Critical thinking

Public speaking (voice)

Analytical mindset

Reliability

Education

Bachelor's degree

Job description

Innodata Inc. seeks detail-oriented Voice Specialists to evaluate AI models via real-time, voice-based conversations. You will roleplay with two models, compare them under consistent scenarios, and document observations across five evaluation dimensions.

The role emphasizes fair testing, recording, and producing a reasoned final choice with a clear rationale grounded in observed performance and guidelines.