Teacher | $36.16/hr Remote

Crossing Hurdles

Canada

On-site

CAD 60,000 - 80,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

A leading tech evaluation company in Canada seeks an AI Evaluation Specialist to assess LLM-generated responses for accuracy and relevance. The ideal candidate will be fluent in French and proficient in English, with a background in analytical fields. Responsibilities include fact-checking, annotating AI response strengths and weaknesses, and ensuring adherence to guidelines. This role requires strong writing skills and attention to detail, making it suitable for candidates with experience in research or analytics.

Qualifications

  • Fluent in French (ILR 5 / CEFR C2) and proficient in English.
  • Experience with LLMs and understanding of their practical applications.
  • Ability to provide structured feedback on AI outputs.
  • Ability to work across varied topics and evolving requirements.

Responsibilities

  • Evaluate LLM-generated responses for accuracy and relevance.
  • Perform fact-checking using reliable sources.
  • Create quality evaluation data by annotating AI responses.
  • Assess reasoning quality and adherence to guidelines.

Skills

Native-level French fluency
Strong English proficiency
Experience with large language models
Excellent writing skills
Strong attention to detail
Analytical thinking

Education

Background in research, policy, analytics, linguistics, or engineering

Job description

  • Evaluate LLM-generated responses for accuracy, relevance, and effectiveness across a wide range of topics.
  • Perform fact-checking using reliable public sources and external tools.
  • Create high-quality human evaluation data by annotating response strengths, gaps, and factual errors.
  • Assess reasoning quality, tone, clarity, and completeness of AI-generated outputs.
  • Ensure responses follow expected conversational behavior and system guidelines.
  • Apply consistent annotations using defined taxonomies, benchmarks, and evaluation frameworks.
Requirements
  • Native-level or near-native fluency in French (ILR 5 / CEFR C2) with strong English proficiency.
  • Proven experience using large language models and understanding real-world LLM use cases.
  • Excellent writing skills with the ability to provide clear, nuanced, and structured feedback.
  • Strong attention to detail and analytical thinking.
  • Ability to work across multiple topics, domains, and evolving requirements.
  • Background in structured analytical fields such as research, policy, analytics, linguistics, or engineering.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Writer | $36.16/hr Remote
Writer | $36.16/hr Remote

Crossing Hurdles • Canada

On-site
CAD 65,000 - 90,000
LLM Evaluation Specialist: French and English
LLM Evaluation Specialist: French and English

Crossing Hurdles • Canada

On-site
CAD 65,000 - 90,000
French-Language LLM Evaluation Specialist
French-Language LLM Evaluation Specialist

Crossing Hurdles • Canada

On-site
CAD 60,000 - 80,000
English Language Specialist - Fully Remote | Upto $50/hr
English Language Specialist - Fully Remote | Upto $50/hr

Mercor • Toronto

Remote
CAD 81,000 - 115,000
English Language Expert
English Language Expert

SME Careers • Calgary

Remote
CAD 34,000 - 62,000
Remote French Gen AI Quality Analyst (Canada)
Remote French Gen AI Quality Analyst (Canada)

Welo Data • Montreal

Remote
CAD 33,000 - 45,000
Remote freelance role
Writer | $45/hr Remote
Writer | $45/hr Remote

Crossing Hurdles • Canada

Remote
USD 25,000 - 44,000
Linguistic AI Auditor - French From Canada
Linguistic AI Auditor - French From Canada

Softechub • Montreal

On-site
CAD 55,000 - 75,000
Content Evaluator | $70/hr Remote
Content Evaluator | $70/hr Remote

Crossing Hurdles • Canada

On-site
CAD 60,000 - 80,000
RLHF Quality Reviewer and Model Evaluator Remote Canada English C1, French C1 CAD 40 to 65 per hour
RLHF Quality Reviewer and Model Evaluator Remote Canada English C1, French C1 CAD 40 to 65 per hour

Corpshore Solutions Corporation • Canada

Hybrid
CAD 34,000 - 55,000
Fully remote across Canada
Flexible weekly hours (20+)
Paid guideline training per programme
+3