Remote AI Media Evaluator: Spot Errors & Improve Models

Outlier; Madison (WI); Remote; USD 20,000 - 29,000; Full time

Benefits offered by this job

Earn up to $18 USD/hr

Flexible hours

Free access to leading LLMs

Weekly payments

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media on a fully remote basis. You will hold live conversations with AI voice models, compare images and videos, and explain what’s off, all on a flexible schedule.

Responsibilities include ranking outputs, writing concise explanations in English, and ensuring quality across audio, video, and image tasks. This role supports research-grade evaluation from home with weekly payments.

Qualifications

• Fluent English with strong, clear written communication.
• Strong attention to detail; able to replay or zoom in when needed.
• Bachelor's degree or higher preferred.
• Comfortable thinking on your feet and explaining decisions aloud.
• Experience writing concise, evidence-based reviews.
• Able to decide when both options have flaws.
• Eligible to apply from Australia, Canada, Ireland, United Kingdom, or the United States.

Responsibilities

• Holding a live, scenario-based conversation in English with AI voice models, then ranking how natural, useful, and clear each sounded.
• Comparing pairs of AI-generated images and videos and judging which one better follows the prompt.
• Inspecting outputs closely, including zooming in and replaying, to spot problems like distorted anatomy, flickering or unnatural motion, text errors, and audio that doesn't match the video.
• Rating each output across dimensions like adherence, quality, and realism, then writing a specific explanation in English that points to what you saw or heard.
• Image evaluations: compare two AI-generated images for the same text prompt and pick the better one, judging how well each follows the instructions, visual quality and visible AI artifacts.
• Video evaluations: rank AI-generated videos against a text prompt, or edited videos against their source clip.
• Video caption labeling: label first-person videos of people or robots doing hands-on tasks, and write precise captions, starting from an AI draft.
• Audio and voice evaluations: rate AI-generated speech on naturalness and pronunciation.

Skills

Fluent English

Attention to detail

Clear written communication

Conversational thinking

Education

Bachelor's degree or above

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media on a fully remote basis. You will hold live conversations with AI voice models, compare images and videos, and explain what’s off, all on a flexible schedule.

Responsibilities include ranking outputs, writing concise explanations in English, and ensuring quality across audio, video, and image tasks. This role supports research-grade evaluation from home with weekly payments.