Remote AI Rater & Evaluator (Flexible Hours, Freelance)

Scale AI; Kentville; Remote; CAD 24,000 - 34,000; Full time

Benefits offered by this job

Remote work

Flexible scheduling

Free access to leading LLMs

Job summary

Outlier, an AI training platform, is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, score outputs for realism and adherence, and explain discrepancies in writing.

Work remotely on your own schedule, with payments weekly. Flexible hours and access to leading LLMs are provided; applicants from multiple countries are welcome to apply as contractors.

Qualifications

• Fluency in English with clear written communication.
• Attention to detail and patience to replay or zoom in to spot issues.
• Ability to think on your feet and discuss realistic scenarios.
• Experience writing concise, evidence-based reviews.
• Ability to make clear decisions when both options have flaws.
• Nice to have: Bachelor’s degree or above.

Responsibilities

• Hold a live, scenario-based conversation in English with AI voice models and rank naturalness, usefulness, and clarity.
• Compare AI-generated images and videos and judge which follows the prompt better.
• Inspect outputs closely to spot issues like distortion, glitches, or misalignment and report findings.
• Rate each output across dimensions and write a concise explanation in English detailing what was observed.

Skills

Fluent English

Written communication

Attention to detail

Decision making

Bachelor’s degree preferred

Education

Bachelor’s degree or higher

Job description

Outlier, an AI training platform, is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, score outputs for realism and adherence, and explain discrepancies in writing.

Work remotely on your own schedule, with payments weekly. Flexible hours and access to leading LLMs are provided; applicants from multiple countries are welcome to apply as contractors.