Remote AI Evaluation Specialist

Scale AI; Vancouver; Hybrid; CAD 24,000 - 34,000; Part time

Benefits offered by this job

Flexible schedule

Free access to leading LLMs

Global expert community

Job summary

Outlier is an AI evaluation platform operated by Scale AI. We seek detail-oriented English speakers to review AI-generated media, roleplay live conversations, and score exchanges on naturalness and usefulness, remotely and on your schedule.

You will compare AI-generated images and videos, assess prompt adherence, and explain issues in writing. This is a freelance, 1099 opportunity with flexible hours and weekly payments.

Qualifications

• Fluent English speaker with clear written communication.
• Strong attention to detail and the patience to replay or zoom in when something seems off.
• Ability to think on your feet and discuss realistic situations clearly in conversation.
• Experience writing short, evidence-based explanations or reviews.
• Ability to make clear decisions when both options have flaws.
• Bachelor’s degree or higher preferred.

Responsibilities

• Hold live, scenario-based English conversations with AI voice models.
• Rank outputs by adherence, quality, and realism and explain your judgment.
• Inspect outputs closely for artifacts like distorted anatomy, flicker, or audio-video mismatch.
• Write concise explanations in English describing what you saw or heard.

Skills

Fluent English

Attention to detail

Clear written communication

Decision making under uncertainty

Adaptability

Education

Bachelor’s degree or above

Job description

Outlier is an AI evaluation platform operated by Scale AI. We seek detail-oriented English speakers to review AI-generated media, roleplay live conversations, and score exchanges on naturalness and usefulness, remotely and on your schedule.

You will compare AI-generated images and videos, assess prompt adherence, and explain issues in writing. This is a freelance, 1099 opportunity with flexible hours and weekly payments.