AI Voice Quality Evaluator

Innodata Inc.; Columbus (OH); On-site; USD 28,000 - 34,000; Full time

Job summary

Innodata Inc. is seeking a Voice Specialist to evaluate AI model conversations in real-time. You will roleplay with two distinct models, compare responses, and provide structured evaluations across five criteria. Attention to detail, listening skills, and clear written reporting are essential.

The role focuses on fair comparisons, documenting error clusters, and delivering a detailed rationale for the final model preference, based on methodical analysis and consistent guidelines.

Qualifications

• Bachelor's degree required.
• Excellent listening and comprehension skills.
• Strong written communication to explain observations clearly.
• Ability to follow structured evaluation guidelines.
• Comfort with roleplaying and speaking naturally.
• Strong analytical and critical-thinking skills.
• Reliability and consistency in completing structured tasks.
• Familiarity with AI assistants and voice-based systems.

Responsibilities

• Review scenario and roleplay conversations with two AI models.
• Conduct comparable conversations across Model A and Model B with consistent approach.
• Maintain similar conversation length across models for fairness.
• Record voice during interactions and assess model quality.
• Evaluate models across five defined evaluation dimensions.
• Identify and categorize error clusters in audio/conversation.
• Compare performance and select the stronger model.
• Provide a detailed rationale with examples from both conversations.

Skills

Attention to detail

Active listening

Written communication

Evaluation guidelines

Roleplay conversations

Critical thinking

Reliability

Conversational systems familiarity

Education

Bachelor's Degree

Job description

Innodata Inc. is seeking a Voice Specialist to evaluate AI model conversations in real-time. You will roleplay with two distinct models, compare responses, and provide structured evaluations across five criteria. Attention to detail, listening skills, and clear written reporting are essential.

The role focuses on fair comparisons, documenting error clusters, and delivering a detailed rationale for the final model preference, based on methodical analysis and consistent guidelines.