Remote AI Quality Evaluator — Spot AI Artifacts & Errors

Outlier; Montreal; Remote; CAD 29,000 - 41,000; Part time

Benefits offered by this job

Flexible hours

Free access to leading LLMs — worth $+

Up to $18 USD/hr pay

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay conversations with AI voice models, rank outputs, and explain in writing what looked or sounded off, remotely on your own schedule.

Responsibilities include evaluating images, videos, captions, and audio, with deadlines and quality standards. This opportunity offers flexible hours and remote work from eligible countries.

Qualifications

• Fluent English with strong, clear written communication.
• Strong attention to detail and ability to focus closely.
• Bachelor's degree or above is nice to have.
• Think on your feet and discuss realistic scenarios clearly.
• Experience writing concise, evidence-based explanations or reviews.
• Able to make clear decisions when both options have flaws.

Responsibilities

• Hold live, scenario-based conversations with AI voice models and rate naturalness, usefulness, and clarity.
• Compare AI-generated images and videos to judge adherence to the prompt.
• Inspect outputs closely, zooming in and replaying to spot issues like distorted anatomy, flicker, or audio-video misalignment.
• Write a clear explanation in English describing what you saw or heard, with evidence.
• Image evaluations: compare two AI-generated images for the same prompt and pick the better one.
• Video evaluations: rank AI-generated videos against a prompt or source clip.
• Video caption labeling: label first-person videos with precise actions and concise captions.
• Audio and voice evaluations: rate speech naturalness and voice match.

Skills

Fluent English

Attention to detail

Clear written communication

Education

Bachelor's degree or above

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay conversations with AI voice models, rank outputs, and explain in writing what looked or sounded off, remotely on your own schedule.

Responsibilities include evaluating images, videos, captions, and audio, with deadlines and quality standards. This opportunity offers flexible hours and remote work from eligible countries.