Voice AI Evaluation Specialist: Real-Time Model Comparisons

Innodata Inc.; Town of Texas (WI); On-site; USD 28,000 - 37,000; Part time

Job summary

Innodata Inc. is seeking a detail-oriented Voice Specialist to evaluate AI model performance through real-time voice conversations. You will roleplay with two models, Model A and Model B, and provide structured evaluations across five criteria.

The role emphasizes careful listening, objective reporting, and adherence to evaluation guidelines. The position is based in the United States and offers an hourly pay range of $20-$27, reflecting experience and qualifications.

Qualifications

• Ability to notice subtle differences in conversational quality.
• Familiarity with AI assistants, voice-based systems or conversational AI is preferred.
• Excellent listening and analytical skills are required.

Responsibilities

• Review an assigned scenario and roleplay a natural conversation with two different AI models.
• Conduct comparable conversations with Model A and Model B with consistent scenario and approach.
• Maintain similar number of turns to ensure a fair comparison.
• Record voice during interactions and evaluate model responses across defined criteria.
• Identify and categorize error clusters in audio or conversation.
• Write a detailed rationale comparing model performance with references to observations.

Skills

Detail-oriented

Excellent listening

Strong written communication

Attention to evaluation guidelines

Fair comparison

Education

Bachelor's Degree

Job description

Innodata Inc. is seeking a detail-oriented Voice Specialist to evaluate AI model performance through real-time voice conversations. You will roleplay with two models, Model A and Model B, and provide structured evaluations across five criteria.

The role emphasizes careful listening, objective reporting, and adherence to evaluation guidelines. The position is based in the United States and offers an hourly pay range of $20-$27, reflecting experience and qualifications.