Remote AI Evaluation Specialist — Flexible Hours (Freelance)

Scale AI; Calgary; Remote; CAD 24,000 - 34,000; Full time

Benefits offered by this job

Flexible schedule

Free access to leading LLMs

Join 900K+ Outlier Experts worldwide

Job summary

Outlier, an AI evaluation platform owned by Scale AI, seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, compare images and videos to judge prompt adherence, and explain what looked or sounded off in writing.

You will rate outputs across adherence, quality, and realism, and work remotely with a flexible schedule. Applicants from multiple countries can apply; payments are weekly for project work, with additional

Qualifications

• Fluent English with clear written communication.
• Excellent attention to detail and patience to replay or zoom in when something seems off.
• Ability to think on your feet and discuss realistic scenarios naturally.
• Experience writing short, evidence-based explanations or reviews.

Responsibilities

• Hold live, scenario-based conversations in English with AI voice models and rank naturalness, usefulness, and clarity.
• Compare AI-generated images and videos and judge adherence to prompts.
• Inspect outputs closely (zoom/replay) to spot issues like distorted anatomy, flickering, text errors, or mismatched audio.
• Write concise explanations in English highlighting observed issues and rate across adherence, quality, and realism.

Skills

Fluent English

Attention to detail

Written communication

Decision making

Live-conversation skills

Education

Bachelor’s degree or above (nice to have)

Job description

Outlier, an AI evaluation platform owned by Scale AI, seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, compare images and videos to judge prompt adherence, and explain what looked or sounded off in writing.

You will rate outputs across adherence, quality, and realism, and work remotely with a flexible schedule. Applicants from multiple countries can apply; payments are weekly for project work, with additional