Remote AI Benchmark Evaluator & Quality Reviewer

Scale AI; Abbotsford; Remote; CAD 17,000 - 29,000; Full time

Benefits offered by this job

Flexible schedule

Remote work from anywhere

Free access to leading LLMs

Global expert community

Job summary

Outlier is an AI training platform owned by Scale AI. We seek detail-oriented English speakers to review AI-generated media, roleplay live conversations with AI voice models, and score exchanges.

You will compare AI-generated images and videos to prompts, and explain what looked or sounded wrong, all remotely and on your own schedule. Join Outlier to help shape how advanced AI systems reason and improve through evaluation, feedback, and detailed reasoning.

Qualifications

• Fluent English with strong written communication.
• Excellent attention to detail and ability to replay or zoom in.
• Ability to think on your feet and discuss realistic scenarios.
• Experience writing short, evidence-based explanations or reviews.
• Clear decision-making when two options have flaws.

Responsibilities

• Hold live conversations in English with AI voice models and rank naturalness and usefulness.
• Compare AI-generated images and videos and judge adherence to prompts.
• Inspect outputs closely to spot issues like distorted anatomy, glitches, text or audio mismatches.
• Write explicit explanations in English describing observed issues and decisions.

Skills

Fluent English

Attention to detail

Clear written communication

Fast learner

Evidence-based reviews

Education

Bachelor’s degree or above

Job description

Outlier is an AI training platform owned by Scale AI. We seek detail-oriented English speakers to review AI-generated media, roleplay live conversations with AI voice models, and score exchanges.

You will compare AI-generated images and videos to prompts, and explain what looked or sounded wrong, all remotely and on your own schedule. Join Outlier to help shape how advanced AI systems reason and improve through evaluation, feedback, and detailed reasoning.