Remote AI Quality Evaluator – Canada

AuraOne Human Data

United States

Remote

USD 17,000 - 24,000

Part time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

AuraOne, Inc. seeks an AI Evaluation Specialist (Canada) for remote, contractor work. You will review AI outputs, assess alignment with workflows and policies, and document corrective steps for training the modeling team.

The role requires experience with multi-page rubrics, clear policy-based reasoning, and reliable weekly availability. Remote work is US-eligible, with emphasis on accurate risk flags and process improvements.

Qualifications

  • Direct working experience in ai evaluation specialist operations on real teams for AI Evaluation Specialist (Canada) work.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the policy or workflow being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Responsibilities

  • Review AI outputs against current ai evaluation specialist operations workflows, playbooks, and firm policy for AI Evaluation Specialist (Canada) assignments.
  • Grade tone, escalation logic, and stakeholder fit on a structured rubric.
  • Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
  • Capture the right next step so the modeling team can train on it.
  • Adjudicate disputed treatments against published playbooks or firm guidance.
  • Maintain reviewer-quality scores in inter-rater calibration cycles.

Skills

Operational review
Policy adherence
Workflow judgment
Stakeholder communication
AI Evaluation specialist operations
AI model evaluation

Job description

AuraOne, Inc. seeks an AI Evaluation Specialist (Canada) for remote, contractor work. You will review AI outputs, assess alignment with workflows and policies, and document corrective steps for training the modeling team.

The role requires experience with multi-page rubrics, clear policy-based reasoning, and reliable weekly availability. Remote work is US-eligible, with emphasis on accurate risk flags and process improvements.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI Workflow Quality Review - Generalist Contractor
Remote AI Workflow Quality Review - Generalist Contractor

AuraOne Human Data • United States

Remote
USD 30,000 - 47,000
Remote AI Image Quality Reviewer & Rubric Specialist
Remote AI Image Quality Reviewer & Rubric Specialist

AuraOne Human Data • United States

Remote
USD 28,000 - 48,000
Remote AI Evaluation & Feedback Specialist
Remote AI Evaluation & Feedback Specialist

Auraone • United States

Remote
USD 34,000 - 83,000
Remote work
Contractor engagement
US-eligible
Remote AI Evaluation Specialist - Task-Completion Quality
Remote AI Evaluation Specialist - Task-Completion Quality

AuraOne Human Data • United States

Remote
USD 34,000 - 62,000
Remote AI Evaluation Rubric Reviewer & Feedback Analyst
Remote AI Evaluation Rubric Reviewer & Feedback Analyst

AuraOne Human Data • United States

Remote
USD 55,000 - 103,000
AI Evaluation Data Analyst — Remotely Review & Calibrate
AI Evaluation Data Analyst — Remotely Review & Calibrate

Auraone • United States

Remote
USD 97,000 - 138,000
Remote AI Evaluation & Rubric Annotation Specialist
Remote AI Evaluation & Rubric Annotation Specialist

AuraOne Human Data • United States

Remote
USD 28,000 - 48,000
Remote AI Evaluation Rubric Reviewer & Feedback Specialist
Remote AI Evaluation Rubric Reviewer & Feedback Specialist

AuraOne Human Data • United States

Remote
USD 34,000 - 55,000
Remote AI Evaluation Lead, Data Operations
Remote AI Evaluation Lead, Data Operations

AuraOne Human Data • United States

Remote
USD 34,000 - 83,000
Remote AI Search Evaluation Specialist
Remote AI Search Evaluation Specialist

AuraOne Human Data • United States

Remote
USD 41,000 - 55,000