AI Media Evaluator - Remote & Flexible Hours

Outlier; Flagstaff (AZ); Remote; USD 20,000 - 29,000; Full time

Benefits offered by this job

Flexible hours

Remote work

Weekly payments via ACH/PayPal

Free access to leading LLMs

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media from home, on your own schedule. You will roleplay live conversations with AI voice models and score outputs for adherence to prompts, then explain your observations in writing.

Tasks include comparing image/video pairs, caption labeling, and audio evaluations, with flexible hours and remote work. Ideal candidates are fluent in English, highly detail-focused, and able to clearly justify decisions.

Qualifications

• Fluent English speaker with clear written communication.
• Strong attention to detail and the patience to replay or zoom in when something seems off.
• Nice to have: Bachelor’s degree or above.
• Comfortable thinking on your feet and explaining decisions in natural conversations.
• Experience writing concise, evidence-based explanations or reviews.
• Able to make clear decisions when both options have flaws.
• Open to applicants from multiple countries including the US.

Responsibilities

• Hold live, scenario-based conversations with AI voice models and rank naturalness and usefulness.
• Compare AI-generated images and videos and judge which follows the prompt better.
• Inspect outputs closely—zoom in, replay, and spot issues like artifacts, misalignment, or audio-video mismatch.
• Write a concise English explanation pointing to observed problems, per task.

Skills

Fluent English

Attention to detail

Decision making

Clear written communication

Evidence-based reviews

Education

Bachelor's degree

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media from home, on your own schedule. You will roleplay live conversations with AI voice models and score outputs for adherence to prompts, then explain your observations in writing.

Tasks include comparing image/video pairs, caption labeling, and audio evaluations, with flexible hours and remote work. Ideal candidates are fluent in English, highly detail-focused, and able to clearly justify decisions.