Remote AI Evaluation Specialist

Outlier (by Scale AI); City Of London; Remote; GBP 11,000 - 15,000; Part time

Job summary

Outlier (by Scale AI) seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model and score exchanges, compare AI-generated images and videos for prompt adherence, and explain in writing what looked or sounded wrong, all on your own schedule, remotely.

You will evaluate image and video outputs, caption labeling, and audio evaluations, providing concise, evidence-based explanations.

Qualifications

• Fluent English with clear written communication.
• Excellent attention to detail and ability to replay or zoom in when something seems off.
• Comfortable thinking on your feet and discussing realistic scenarios naturally.
• Experience writing short, evidence-based explanations or reviews.
• Willingness to make a clear decision when both options have flaws.
• Nice to have: pursuing or holding a Bachelor’s degree or above.

Responsibilities

• Holding live, scenario-based conversations in English with AI voice models and ranking naturalness and usefulness.
• Comparing pairs of AI-generated images and videos to judge adherence to prompts.
• Inspecting outputs for issues like distorted anatomy, flickering, text errors, and audio-video inconsistencies.
• Writing concise explanations in English detailing observations and reasoning.

Skills

Fluent English

Attention to detail

Strong written communication

Thinking on your feet

Clear decision making

Education

Bachelor’s degree or higher

Job description

Outlier (by Scale AI) seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model and score exchanges, compare AI-generated images and videos for prompt adherence, and explain in writing what looked or sounded wrong, all on your own schedule, remotely.

You will evaluate image and video outputs, caption labeling, and audio evaluations, providing concise, evidence-based explanations.