Elite AI Voice Evaluation Specialist

Innodata Inc.; New Hampshire; On-site; USD 28,000 - 37,000; Part time

Job summary

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations. You will interact with two different AI models, compare responses, and provide structured evaluations based on five criteria.

The role focuses on fair comparisons, identifying error clusters, and documenting observations with a detailed final rationale that supports the preferred model choice.

Qualifications

• Bachelor's degree required.
• Strong attention to detail and listening skills.
• Ability to follow evaluation guidelines and provide objective analyses.
• Familiarity with AI assistants, voice-based AI, or conversational systems is a plus.

Responsibilities

• Review an assigned scenario and roleplay a natural conversation with two different AI models.
• Conduct comparable conversations with Model A and Model B with consistent scenario and approach.
• Maintain approximately the same number of turns with each model for fairness.
• Record your voice during interactions and assess model responses.
• Evaluate each model across five defined evaluation dimensions.
• Identify and categorize relevant error clusters in audio or conversation.
• Compare performance and select the stronger model based on criteria.
• Write a detailed rationale referencing observed differences.
• Apply evaluation guidelines consistently across scenarios.

Skills

Attention to detail

Listening skills

Written communication

Evaluation guidelines

Critical thinking

Education

Bachelor's Degree

Job description

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations. You will interact with two different AI models, compare responses, and provide structured evaluations based on five criteria.

The role focuses on fair comparisons, identifying error clusters, and documenting observations with a detailed final rationale that supports the preferred model choice.