AI Voice Evaluation Analyst — Real-Time Model Compare

Innodata Inc.; New Jersey; On-site; USD 28,000 - 37,000; Full time

Job summary

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations.

You will interact with two different AI models, perform roleplays, and provide structured evaluations across five dimensions. Responsibilities include maintaining comparable turns, recording voice, identifying error clusters, and delivering a final rationale that supports selecting the stronger model.

Qualifications

• Bachelor's degree required.
• Excellent listening and comprehension skills.
• Strong written communication for objective reporting.
• Ability to follow detailed evaluation guidelines.
• Comfort speaking naturally and roleplaying scenarios.
• Ability to compare two interactions fairly.
• Familiarity with AI assistants, voice-based AI, or conversational systems.

Responsibilities

• Review assigned scenarios and roleplay a natural conversation with two different AI models.
• Conduct comparable conversations with Model A and Model B, keeping scenario and approach consistent.
• Maintain approximately the same number of conversational turns with each model.
• Record your voice during each interaction and evaluate model responses.
• Evaluate across five defined evaluation dimensions and identify error clusters.
• Compare model performance and select the stronger overall option.
• Write a detailed rationale referencing specific observations.

Skills

Attention to detail

Listening skills

Written communication

Evaluation guidelines

Critical thinking

Education

Bachelor's degree

Job description

Innodata Inc. is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations.

You will interact with two different AI models, perform roleplays, and provide structured evaluations across five dimensions. Responsibilities include maintaining comparable turns, recording voice, identifying error clusters, and delivering a final rationale that supports selecting the stronger model.