AI Voice Quality Evaluator

Innodata Inc.; Orangeburg (SC); On-site; USD 28,000 - 34,000; Part time

Job summary

Innodata Inc. seeks detail-oriented Voice Specialists to evaluate AI model performance through real-time, voice-based conversations. You will compare two models, Model A and Model B, and provide structured evaluations across five criteria, identifying error clusters and selecting the stronger performer.

Requires a Bachelor’s degree and sharp listening skills. The role focuses on consistent conversations, sound recording of interactions, and thorough rationale highlighting observations from both

Qualifications

• Bachelor's degree required.
• Strong listening and comprehension skills.
• Ability to explain observations clearly and objectively.

Responsibilities

• Review an assigned scenario and roleplay a natural conversation with two different AI models.
• Conduct comparable conversations with Model A and Model B, keeping the scenario and approach consistent.
• Maintain roughly the same number of turns for each model to ensure fairness.
• Record your voice during each interaction and evaluate model responses.
• Evaluate each model across five defined evaluation dimensions.
• Identify and categorize error clusters that may occur in audio or conversation.
• Compare performance and select the model with stronger overall performance.
• Write a detailed rationale referencing observations from both conversations.
• Apply guidelines consistently across scenarios and interactions.

Skills

Listening skills

Attention to detail

Written communication

Education

Bachelor's degree

Job description

Innodata Inc. seeks detail-oriented Voice Specialists to evaluate AI model performance through real-time, voice-based conversations. You will compare two models, Model A and Model B, and provide structured evaluations across five criteria, identifying error clusters and selecting the stronger performer.

Requires a Bachelor’s degree and sharp listening skills. The role focuses on consistent conversations, sound recording of interactions, and thorough rationale highlighting observations from both