AI Response Evaluator — Remote, Flexible Hours

Outlier; Norman (OK); Remote; USD 20,000 - 29,000; Full time

Benefits offered by this job

Earn Up to $18 USD/hr

Work flexibly from anywhere

Free access to leading LLMs

Missions surge pricing

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model and score exchanges, compare generated images and videos to judge which follows the prompt better, and explain in writing what looked or sounded off, all remotely and on your own schedule.

At Outlier, your expertise helps shape AI systems and improve performance in real-world use.

Qualifications

• Fluent English speaker with strong, clear written communication.
• Strong attention to detail, patient to replay or zoom in when something seems off.
• Nice to have: Bachelor’s degree or above.
• Comfortable thinking on your feet and speaking through realistic situations.
• Experience writing short, evidence-based explanations or reviews.
• Able to make a clear decision when both options have flaws.
• Eligible from: Australia, Canada, Ireland, United Kingdom, United States.

Responsibilities

• Holding a live, scenario-based conversation in English with AI voice models and ranking naturalness, usefulness, and clarity.
• Comparing AI-generated images and videos and judging which better follows the prompt.
• Inspecting outputs closely to spot problems like distorted anatomy, flickering or unnatural motion, text errors, and audio-video mismatch.
• Rating outputs across dimensions and writing a specific English explanation of observations.

Skills

Fluent English

Attention to detail

Clear written communication

Decision making with imperfect options

Ability to explain observations in écr

Education

Bachelor’s degree or above

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model and score exchanges, compare generated images and videos to judge which follows the prompt better, and explain in writing what looked or sounded off, all remotely and on your own schedule.

At Outlier, your expertise helps shape AI systems and improve performance in real-world use.