Voice AI Quality Evaluator

Innodata Inc.; Town of Florida (NY); On-site; USD 28,000 - 37,000; Full time

Job summary

Innodata Inc. in the United States is seeking detail-oriented Voice Specialists to evaluate AI model performance through real-time, voice-based conversations.

You will interact with two different AI models, compare their responses, and produce structured evaluations based on defined quality criteria. The role requires completing balanced conversations, recording your voice, identifying error clusters, and delivering a detailed rationale for a final model preference.

Qualifications

• Bachelor's degree and strong attention to detail.
• Excellent listening and comprehension skills.
• Strong written communication for objective reporting.
• Ability to follow detailed evaluation guidelines consistently.
• Familiarity with AI assistants or voice-based systems is a plus.

Responsibilities

• Review scenarios and roleplay conversations with two AI models.
• Record and compare model responses across five evaluation dimensions.
• Identify and categorize error clusters in audio or conversation.
• Write a detailed rationale comparing Model A and Model B.
• Apply evaluation guidelines consistently across scenarios.

Skills

Attention to detail

Excellent listening skills

Strong written communication

Follow evaluation guidelines

Fair, objective analysis

Reliability & consistency

Education

Bachelor's Degree

Job description

Innodata Inc. in the United States is seeking detail-oriented Voice Specialists to evaluate AI model performance through real-time, voice-based conversations.

You will interact with two different AI models, compare their responses, and produce structured evaluations based on defined quality criteria. The role requires completing balanced conversations, recording your voice, identifying error clusters, and delivering a detailed rationale for a final model preference.