Remote AI Evaluation Specialist

Scale AI; Laval; Remote; CAD 17,000 - 29,000; Full time

Job summary

Outlier is an AI training platform operated by Scale AI. We seek detail-oriented English speakers to review AI-generated media by roleplaying live conversations with an AI voice model, comparing prompt adherence in images and videos, and writing precise explanations of observed issues.

You can work remotely on your own schedule. Responsibilities include rating outputs across adherence, quality, and realism, with a focus on clarity in explanations.

Qualifications

• Fluent English with clear written communication.
• Excellent attention to detail and listening skills.
• Ability to think on your feet and explain decisions clearly.
• Experience writing concise, evidence-based reviews.
• Ability to decide clearly when options have flaws.

Responsibilities

• Hold live, scenario-based conversations in English with AI voice models and rank naturalness, usefulness, and clarity.
• Compare AI-generated images and videos to judge which better follows the prompt.
• Inspect outputs for distortions, flickering, or misalignment and note issues in writing.
• Write explanations in English detailing what you saw or heard, focusing on adherence, quality, and realism.
• Image evaluations: compare two AI-generated images for the same prompt and judge quality.
• Video evaluations: rank AI-generated videos against a text prompt.

Skills

Fluent English

Attention to detail

Verbal reasoning on the fly

Concise explanations

Decision making under ambiguity

Bachelor’s degree preferred

Education

Bachelor’s degree or above

Job description

Outlier is an AI training platform operated by Scale AI. We seek detail-oriented English speakers to review AI-generated media by roleplaying live conversations with an AI voice model, comparing prompt adherence in images and videos, and writing precise explanations of observed issues.

You can work remotely on your own schedule. Responsibilities include rating outputs across adherence, quality, and realism, with a focus on clarity in explanations.