Remote English Conversational AI Evaluator

Scale AI; United States; Remote; USD 17,000 - 24,000; Full time

Job summary

Outlier, an AI training platform, is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, score exchanges, and compare generated images and videos to judge which follows the prompt better, with written explanations.

You will inspect outputs for distortions or mismatches, rate adherence and quality, and provide concise in-writing feedback in English.

Qualifications

• Fluent English with clear written communication.
• Strong attention to detail and the patience to replay or zoom in when something seems off.
• Able to think on your feet and discuss realistic situations in a natural way.
• Experience writing short, evidence-based explanations or reviews.
• Willingness to make clear decisions when both options have flaws.
• Bachelor's degree or higher preferred.

Responsibilities

• Hold live, scenario-based conversations in English with AI voice models and rank naturalness, usefulness, and clarity.
• Compare pairs of AI-generated images and videos and judge which better follows the prompt.
• Inspect outputs closely, including zooming in and replaying, to spot problems like distortions or audio-video misalignment.
• Write detailed explanations in English describing what you saw or heard.

Skills

Fluent English

Attention to detail

Clear communication

Evidence-based writing

Decision making

Education

Bachelor's degree or higher

Job description

Outlier, an AI training platform, is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, score exchanges, and compare generated images and videos to judge which follows the prompt better, with written explanations.

You will inspect outputs for distortions or mismatches, rate adherence and quality, and provide concise in-writing feedback in English.