AI Response Evaluator: Remote & Flexible Hours

Outlier; Des Moines (IA); Remote; USD 15,000 - 25,000; Full time

Benefits offered by this job

Flexible schedule

Remote work

Free access to leading LLMs

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, score exchanges, and compare generated images and videos to judge prompt adherence. You will explain in writing what looked or sounded wrong, remotely and on your own schedule.

Join Outlier's evaluation loop to help shape how advanced AI systems reason and learn, turning frontier research into practical improvements for real-world AI performance.

Qualifications

• Fluent English with clear written communication and natural conversational ability.
• Strong attention to detail and patience to replay or zoom in on details.
• Experience writing short, evidence-based explanations or reviews.

Responsibilities

• Hold live, scenario-based conversations with AI voice models and rank naturalness and clarity.
• Compare AI-generated images and videos to judge adherence to prompts.
• Inspect outputs for issues like distorted anatomy, flickering, or audio-video mismatch; provide written explanations.
• Rate outputs across dimensions and provide concise English rationale.

Skills

Fluent English

Attention to detail

Communication

Evidence-based writing

Education

Bachelor's degree or above

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, score exchanges, and compare generated images and videos to judge prompt adherence. You will explain in writing what looked or sounded wrong, remotely and on your own schedule.

Join Outlier's evaluation loop to help shape how advanced AI systems reason and learn, turning frontier research into practical improvements for real-world AI performance.