Remote AI Response Evaluator (English)

Scale AI; Calgary; Remote; CAD 17,000 - 29,000; Full time

Benefits offered by this job

Flexible schedule

Free AI tools access

Global community

Collaborative projects

Job summary

Outlier, an AI evaluation platform owned by Scale AI, seeks detail-oriented English speakers to review AI-generated media, engage in live conversations with AI voice models, and rank outputs based on adherence and quality. You’ll provide written explanations and operate remotely with flexible hours.

Typical tasks include image and video evaluations, caption labeling, and audio assessments, with payment as a freelance contractor. Applicants from multiple countries are welcome.

Qualifications

• Fluent English with clear written communication.
• Strong attention to detail; patient and observant.
• Comfortable speaking naturally in everyday conversations.
• Experience writing evidence-based reviews.
• Ability to decide when both options have flaws.

Responsibilities

• Hold live English conversations with AI voice models and rank naturalness.
• Compare AI-generated images/videos and judge prompt adherence.
• Inspect outputs for distortions, flicker, text errors, and audio-visual mismatches.
• Write explanations highlighting what you saw or heard.

Skills

Fluent English

Attention to detail

Clear written explanations

Conversational thinking

Independent time management

Education

Bachelor’s degree or above (nice to have)

Job description

Outlier, an AI evaluation platform owned by Scale AI, seeks detail-oriented English speakers to review AI-generated media, engage in live conversations with AI voice models, and rank outputs based on adherence and quality. You’ll provide written explanations and operate remotely with flexible hours.

Typical tasks include image and video evaluations, caption labeling, and audio assessments, with payment as a freelance contractor. Applicants from multiple countries are welcome.