Voice AI Quality Analyst

Innodata Inc.; Delaware; On-site; USD 28,000 - 37,000; Full time

Job summary

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI model performance through real-time voice conversations. You will interact with two models using the same scenario, compare responses, and provide structured evaluations across five criteria.

The role emphasizes fairness, consistency, and clear, objective reporting of findings. The position requires a Bachelor's Degree and strong listening, analytical, and written communication skills.

Qualifications

• Bachelor's Degree required.
• Detail-oriented with strong listening skills.
• Experience with AI assistants or conversational systems preferred.
• Ability to compare two interactions fairly and without bias.
• Excellent written communication to explain observations clearly and objectively.
• Comfort speaking naturally and roleplaying different conversational scenarios.

Responsibilities

• Review an assigned scenario and roleplay a natural conversation with two different AI models.
• Conduct comparable conversations with Model A and Model B, ensuring consistency in scenario and approach.
• Maintain roughly equal conversational turns to support a fair comparison.
• Record your voice during each interaction and assess model responses.
• Evaluate models across five defined evaluation dimensions.
• Identify and categorize relevant error clusters observed in audio or conversation.
• Compare performance and select the model with stronger overall performance.
• Write a detailed rationale referencing specific observations from both conversations.
• Apply evaluation guidelines consistently across scenarios.

Skills

Attention to detail

Listening skills

Written communication

Evaluation guidelines

Roleplay conversations

Education

Bachelor's Degree

Job description

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI model performance through real-time voice conversations. You will interact with two models using the same scenario, compare responses, and provide structured evaluations across five criteria.

The role emphasizes fairness, consistency, and clear, objective reporting of findings. The position requires a Bachelor's Degree and strong listening, analytical, and written communication skills.