AI Evaluation Specialist - Remote & Flexible Hours

Scale AI

St. Albert

Remote

CAD 24,000 - 34,000

Full time

3 days ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Free access to leading LLMs

Join 900K+ Outlier Experts across 52+‑

Job summary

Scale AI's Outlier platform seeks detail-oriented English speakers to review AI-generated media, roleplay live conversations with an AI voice model, and score each exchange for naturalness and usefulness. You will compare AI-generated images and videos to judge adherence to prompts, identify issues such as distortions or misalignment, and write precise explanations in English describing observations.

This is a flexible, remote freelance opportunity with weekly payments and potential incentives

Qualifications

Fluent English with strong written communication.
Excellent attention to detail and ability to replay or zoom in on content.
Ability to think on your feet and discuss realistic scenarios clearly.
Experience writing short, evidence-based explanations or reviews.
Capable of making clear decisions when options have flaws.
Nice to have a bachelor’s degree or above.

Responsibilities

Hold live, scenario-based conversations in English with AI voice models and rank naturalness and usefulness.
Compare pairs of AI-generated images/videos and judge prompt adherence.
Inspect outputs closely to spot issues like distorted anatomy, flickering, or misalignment.
Write a concise explanation in English detailing what you saw or heard and why it mattered.