AI Voice Evaluation Specialist - 3 month contract

Innodata Inc.

Canada

Remote

CAD 28,000 - 34,000

Full time

6 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Innodata Inc. in Canada is seeking detail-oriented Voice Specialists to evaluate AI models through real-time, voice-based conversations. You will compare two models using the same scenario and provide structured evaluations across five dimensions.

This 3-month, 40-hour/week W2 contract emphasizes fair comparison, rigorous observation, and clear documentation. You will record responses, identify error patterns, and write a detailed rationale for the final choice.

Qualifications

  • Detail-oriented role evaluating AI model performance.
  • Strong written communication and objective reporting.
  • Familiarity with AI assistants or conversational systems preferred.

Responsibilities

  • Review an assigned scenario and roleplay conversations with two AI models.
  • Conduct comparable conversations with Model A and Model B with consistent approach.
  • Record conversations and assess model responses across defined criteria.
  • Identify error clusters and document observations.
  • Write a detailed rationale for the final model preference.

Skills

Attention to detail
Active listening
Analytical thinking
Written communication

Job description

Remote - Canada

Innodata(Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked.Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale.We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

About this Role:

This role is a 3-month, W2, full-time contract (40 hours per week.)

Scope of the Role:

We are looking for detail-oriented Voice Specialists to evaluate the performance of AI models through real-time, voice-based conversations. In this role, you will interact with two different AI models using the same assigned scenario, carefully compare their responses, and provide structured evaluations based on defined quality criteria.

The goal of the evaluation is to create a fair, consistent comparison between models and identify which model delivers the stronger overall conversational experience.

What You’ll Own:
  • Review an assigned scenario and roleplay a natural conversation with two different AI models.
  • Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible.
  • Maintain approximately the same number of conversational turns with each model to support a fair comparison.
  • Record your voice during each interaction and pay close attention to the quality and behavior of the model's responses.
  • Evaluate each model across five defined evaluation dimensions.
  • Identify and categorize relevant error clusters that may occur in the audio or conversation.
  • Compare the performance of Model A and Model B based on the evaluation criteria and your observations.
  • Select the model that demonstrated the stronger overall performance according to the defined evaluation framework.
  • Write a detailed rationale explaining your final preference, referencing specific examples and observations from both conversations.
  • Apply evaluation guidelines consistently across different scenarios and model interactions
You’ll Thrive in This Role If You Have
  • Strong attention to detail and ability to notice subtle differences in conversational quality.
  • Excellent listening and comprehension skills.
  • Strong written communication skills, particularly the ability to explain observations clearly and objectively.
  • Ability to consistently follow detailed evaluation guidelines.
  • Comfort speaking naturally and role-playing different conversational scenarios.
  • Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation.
  • Strong critical-thinking and analytical skills.
  • Reliability and consistency when completing structured evaluation tasks.
  • Familiarity with AI assistants, voice-based AI, or conversational systems

The expected hourly salary range for this position is $20-$25 CAD p/hour, based on experience, skills, and qualifications.

Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission's guide at https://consumer.ftc.gov/articles/job-scams.

As set forth in Innodata Inc.'s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Voice Evaluation Specialist
AI Voice Evaluation Specialist

Jobgether • Canada

Remote
CAD 28,000 - 34,000
Remote work within Canada
40-hour work week
3-month contract
AI Voice Quality Evaluator - Contract, Remote
AI Voice Quality Evaluator - Contract, Remote

Innodata Inc. • Canada

Remote
CAD 28,000 - 34,000
Generative AI Specialist - Bilingual (French and English)
Generative AI Specialist - Bilingual (French and English)

Innodata Inc. • Calgary

Remote
CAD 24,796 - 33,062
AI Voice Recording Contributor - French (Canada)
AI Voice Recording Contributor - French (Canada)

RWS • Trois-Rivières

Remote
CAD 25,000 - 36,000
Flexible, remote work
Timely payments
Opportunity to contribute to AI tech
+1
AI Voice Recording Contributor - French (Canada)
AI Voice Recording Contributor - French (Canada)

RWS • Quebec

Remote
CAD 26,000 - 37,000
Flexible, remote work
Timely payments
Access to future AI projects
+1
AI Voice Recording Contributor - French (Canada)
AI Voice Recording Contributor - French (Canada)

RWS • Outaouais

Remote
CAD 27,000 - 35,000
Flexible remote work
Timely payments
Cutting-edge AI projects
+2
AI Agent Engineer, Client Facing
AI Agent Engineer, Client Facing

Urban Ridge Supplies • Toronto

On-site
CAD 120,000 - 160,000
Equity
Private health insurance
RRSP
+1
British English Voice Expert
British English Voice Expert

Jobgether • Canada

Remote
CAD 78,000 - 126,000
Fully remote
12-month contract
Part-time ~20h/wk
+1
AI Voice Recording Contributor - French (Canada)
AI Voice Recording Contributor - French (Canada)

RWS Group • Quebec

Remote
CAD 25,000 - 36,000
Flexible, remote work
Timely payments
Opportunity to contribute to cutting‑e
+2
Linguistic AI Auditor - French From Canada
Linguistic AI Auditor - French From Canada

RWS • Quebec

On-site
CAD 39,000 - 55,000
Flexible schedule
Competitive rates
Future project opportunities
+1