AI Voice Evaluation Specialist

Jobgether

Canada

Remote

CAD 28,000 - 34,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Remote work within Canada
40-hour work week
3-month contract

Job summary

Jobgether, listed on behalf of a partner company, is seeking an AI Voice Evaluation Specialist based in Canada to assess two AI models via real-time voice conversations, using identical scenarios to compare performance and determine which system delivers stronger user experiences. The role offers remote work within Canada, a 40-hour week, and a 3-month contract, with hourly pay of CAD 20–25.

A Bachelor's degree is required, and candidates should have excellent listening, analytical, and written

Qualifications

  • Bachelor's degree or equivalent educational background required.
  • Excellent listening and analytical abilities.
  • Strong written communication skills with attention to detail.
  • Ability to follow structured evaluation frameworks and guidelines.

Responsibilities

  • Review assigned scenarios and conduct natural, voice-based roleplay conversations with two different AI models.
  • Interact with Model A and Model B using identical scenarios to ensure fair comparison.
  • Record voice interactions and assess quality, accuracy, relevance, and behavior of responses.
  • Evaluate models across predefined quality dimensions and document observations.
  • Identify and categorize error patterns in audio interactions and conversations.
  • Prepare detailed rationales supporting final evaluations with concrete examples.

Education

Bachelor's degree or equivalent

Job description

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for an AI Voice Evaluation Specialist based in Canada.

Join an innovative AI-focused environment as an AI Voice Evaluation Specialist, where you will contribute to improving the quality, reliability, and performance of next-generation conversational AI systems. In this role, you will interact directly with two different AI models through real-time voice conversations, using identical scenarios to assess and compare their performance. You will evaluate conversational quality, identify potential errors, and analyze how effectively each model responds to different situations. Your observations and structured assessments will help determine which systems deliver the strongest and most natural user experiences. This opportunity is ideal for someone with excellent listening skills, strong analytical abilities, and a keen interest in artificial intelligence and voice-based technologies. You will work independently within a structured evaluation framework, ensuring consistency, objectivity, and accuracy throughout each assessment.

Accountabilities:
  • Review assigned scenarios and conduct natural, voice-based roleplay conversations with two different AI models.
  • Interact with Model A and Model B using the same scenarios, maintaining consistent conversational approaches and conditions to ensure fair comparisons.
  • Keep approximately the same number of conversational turns with each model to support balanced and reliable evaluations.
  • Record voice interactions and carefully assess the quality, accuracy, relevance, and overall behavior of each model\'s responses.
  • Evaluate both AI models across five predefined quality dimensions, following established assessment criteria and evaluation guidelines.
  • Identify, document, and categorize relevant error patterns and clusters occurring during audio interactions or conversations.
  • Compare the performance of both models using structured evaluation frameworks and observations gathered throughout the interactions.
  • Determine which model demonstrates stronger overall performance based on the defined quality standards and evaluation criteria.
  • Prepare detailed written rationales supporting final evaluations, referencing specific conversational examples and observations from both models.
  • Apply evaluation guidelines consistently across different scenarios and interactions to ensure objective, accurate, and dependable results.
  • Maintain high standards of documentation, attention to detail, and evaluation consistency throughout all assigned tasks.
Requirements:
  • Bachelor\'s degree or equivalent relevant educational background.
  • Excellent attention to detail, with the ability to identify subtle differences in conversational quality, response accuracy, and AI model behavior.
  • Strong listening and comprehension skills, including the ability to understand spoken interactions and recognize inconsistencies or errors.
  • Excellent written communication skills, with the ability to document findings clearly, objectively, and accurately.
  • Ability to follow detailed instructions, structured evaluation frameworks, and predefined quality assessment guidelines.
  • Confidence in speaking naturally and participating in roleplay conversations across different scenarios.
  • Strong analytical and critical-thinking abilities, with the capacity to compare two interactions fairly and identify meaningful performance differences.
  • Objectivity and impartiality when evaluating AI-generated responses, without allowing personal preferences to influence final assessments.
  • High levels of reliability, consistency, and time management when completing repetitive and structured evaluation assignments.
  • Genuine interest in artificial intelligence, conversational AI assistants, voice technologies, or emerging AI applications.
  • Ability to work independently while maintaining accuracy and quality throughout the evaluation process.
Benefits:
  • Competitive hourly compensation ranging from CAD $20 to $25 per hour, depending on experience, skills, and qualifications.
  • Full-time contract opportunity for an initial period of 3 months.
  • 40-hour working week.
  • Remote working arrangement within Canada.
  • Opportunity to contribute directly to the evaluation and improvement of advanced AI voice technologies.
  • Hands-on exposure to conversational AI systems and emerging artificial intelligence applications.
  • Opportunity to develop practical experience in AI model evaluation, voice interaction analysis, and structured quality assessment.
  • Collaborative, technology-focused working environment with clearly defined evaluation processes and quality standards.

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Voice Evaluation Specialist - 3 month contract
AI Voice Evaluation Specialist - 3 month contract

Innodata Inc. • Canada

Remote
CAD 28,000 - 34,000
British English Voice Expert
British English Voice Expert

Jobgether • Canada

Remote
CAD 78,000 - 126,000
Fully remote
12-month contract
Part-time ~20h/wk
+1
AI Voice Quality Evaluator - Contract, Remote
AI Voice Quality Evaluator - Contract, Remote

Innodata Inc. • Canada

Remote
CAD 28,000 - 34,000
AI Agent Engineer, Client Facing
AI Agent Engineer, Client Facing

Urban Ridge Supplies • Toronto

On-site
CAD 120,000 - 160,000
Equity
Private health insurance
RRSP
+1
AI Voice Recording Contributor - French
AI Voice Recording Contributor - French

AI Trainer HQ • Canada

Hybrid
CAD 26,000 - 37,000
AI Engineer (Agentic Systems)
AI Engineer (Agentic Systems)

Jobgether • Canada

On-site
CAD 120,000 - 180,000
Competitive compensation
Challenging, innovative projects
Culture of accountability, autonomy, &
Engineering Manager, Voice AI
Engineering Manager, Voice AI

AItoolnavio • Canada

Remote
CAD 190,000 - 250,000
Salary CAD 190k-250k
Equity opportunities
Fully remote Canada/US
+5
Engineering Manager, Voice AI
Engineering Manager, Voice AI

Jobgether • Canada

Remote
CAD 190,000 - 250,000
Equity opportunities
Fully remote work (Canada/US)
In-person team gatherings
+4
AI Voice Recording Contributor - French (Canada)
AI Voice Recording Contributor - French (Canada)

RWS • Outaouais

Remote
CAD 27,000 - 35,000
Flexible remote work
Timely payments
Cutting-edge AI projects
+2
AI Voice Recording Contributor - French (Canada)
AI Voice Recording Contributor - French (Canada)

RWS • Quebec

Remote
CAD 26,000 - 37,000
Flexible, remote work
Timely payments
Access to future AI projects
+1