Senior AI Voice Quality Evaluator

Innodata Inc.; Georgia; On-site; USD 28,000 - 34,000; Full time

Job summary

Innodata Inc. seeks detail-oriented Voice Specialists to evaluate AI model performance through real-time, voice-based conversations.

You will interact with two different AI models, Model A and Model B, and provide structured evaluations across five dimensions to determine which model offers the strongest conversational experience. The role requires careful, consistent roleplay, listening, and the ability to document error clusters and rationale for model preference.

Qualifications

• Bachelor's degree.
• Excellent listening and comprehension skills.
• Ability to follow detailed evaluation guidelines.

Responsibilities

• Review an assigned scenario and roleplay a natural conversation with two different AI models.
• Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible.
• Maintain approximately the same number of conversational turns with each model to support a fair comparison.
• Record your voice during each interaction and evaluate the model's responses.
• Evaluate each model across five defined evaluation dimensions.
• Identify and categorize relevant error clusters that may occur in the audio or conversation.
• Compare the performance of Model A and Model B based on the evaluation criteria and observations.
• Select the model with stronger overall performance and write a detailed rationale referencing observations from both conversations.

Skills

Detail oriented

Listening skills

Written communication

Critical thinking

Education

Bachelor's Degree

Job description

Innodata Inc. seeks detail-oriented Voice Specialists to evaluate AI model performance through real-time, voice-based conversations.

You will interact with two different AI models, Model A and Model B, and provide structured evaluations across five dimensions to determine which model offers the strongest conversational experience. The role requires careful, consistent roleplay, listening, and the ability to document error clusters and rationale for model preference.