Remote AI Evaluation & Quality Specialist

Scale AI; Lethbridge; Remote; CAD 24,000 - 34,000; Part time

Benefits offered by this job

Flexible schedule

Free access to leading LLMs

Global community of experts

Job summary

Outlier, operated by Scale AI, offers a freelance evaluation role reviewing AI-generated media and rating conversations with AI voice models. You work remotely on your own schedule, focusing on accuracy and detailed written explanations.

Ideal candidates are fluent English speakers with keen attention to detail and the ability to decide clearly when faced with imperfect options. A Bachelor's degree or higher is a plus.

Qualifications

• Fluent English with strong written communication and ability to describe observations clearly.
• Excellent attention to detail and patience to replay or zoom in when something seems off.
• Comfortable thinking on your feet and describing realistic scenarios in a natural way.
• Experience writing concise, evidence-based explanations or reviews.
• Able to make clear decisions when both options have flaws.

Responsibilities

• Hold live, scenario-based conversations in English with AI voice models and rate naturalness, usefulness, and clarity.
• Compare AI-generated images and videos and judge which better follows the prompt.
• Inspect outputs closely (zoom in, replay) to spot issues like distorted anatomy, flickering, text or audio mismatch.
• Write a specific English explanation referencing observed information and reasoning.

Skills

Fluent English

Attention to detail

Clear written communication

Experience writing explanations

Decision making under trade-offs

Conversational thinking

Education

Bachelor’s degree or above

Job description

Outlier, operated by Scale AI, offers a freelance evaluation role reviewing AI-generated media and rating conversations with AI voice models. You work remotely on your own schedule, focusing on accuracy and detailed written explanations.

Ideal candidates are fluent English speakers with keen attention to detail and the ability to decide clearly when faced with imperfect options. A Bachelor's degree or higher is a plus.