AI Voice Evaluation Analyst

Innodata Inc.; Alaska; On-site; USD 28,000 - 37,000; Full time

Job summary

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations. You will interact with two AI models using the same scenario, compare their responses, and provide structured evaluations based on five defined dimensions.

The role requires recording your voice, maintaining balance across conversations, identifying error clusters, and writing a detailed rationale referencing observed examples from both conversations.

Qualifications

• Bachelor's degree or equivalent experience.
• Exceptional listening, attention to detail, and written communication skills.
• Ability to follow detailed evaluation guidelines and remain objective.

Responsibilities

• Review an assigned scenario and roleplay a natural conversation with two AI models.
• Conduct comparable conversations with Model A and Model B, ensuring consistency across scenarios.
• Maintain approximately the same number of conversational turns across models.
• Record your voice during each interaction and assess the quality of the model's responses.
• Identify and categorize relevant error clusters that may occur in audio or conversation.
• Compare model performance and select the stronger overall option based on evaluation criteria.
• Write a detailed rationale with concrete examples from both conversations.
• Apply evaluation guidelines consistently across scenarios and model interactions.

Skills

Active listening

Attention to detail

Analytical thinking

Written communication

Education

Bachelor's Degree

Job description

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations. You will interact with two AI models using the same scenario, compare their responses, and provide structured evaluations based on five defined dimensions.

The role requires recording your voice, maintaining balance across conversations, identifying error clusters, and writing a detailed rationale referencing observed examples from both conversations.