Remote AI Evaluation Pro — Spot What AI Gets Wrong

Outlier; Bozeman (MT); Remote; USD 20,000 - 29,000; Full time

Benefits offered by this job

Flexible work hours

Remote work from anywhere

Free access to leading LLMs

Job summary

Outlier seeks detail-oriented English speakers to review AI-generated media on a fully remote, flexible schedule. You will roleplay live conversations with an AI voice model, score exchanges, and compare AI-generated images and videos to judge which follows prompts better.

You will explain what looked or sounded wrong in writing, remotely and at your own pace. At Outlier, your work helps shape how advanced AI systems reason and improve, contributing to real-world AI performance and providing

Qualifications

• Fluent English with clear written communication.
• Strong attention to detail and ability to replay or zoom in when something seems off.
• Pursuing or holding a Bachelor’s degree or higher is a plus.
• Able to think on your feet and discuss realistic scenarios naturally.
• Experience writing short, evidence-based explanations or reviews.

Responsibilities

• Holding live conversations in English with AI voice models, then ranking naturalness and clarity.
• Comparing AI-generated images and videos to judge prompt adherence.
• Inspecting outputs closely to spot problems like artifacts, flickering, or mismatched audio.
• Rating outputs across dimensions and writing explanations in English detailing observations.

Skills

Fluent English

Attention to detail

Comfortable thinking on your feet

Experience writing explanations

Clear written communication

Education

Bachelor's degree or above (pursuing/held)

Job description

Outlier seeks detail-oriented English speakers to review AI-generated media on a fully remote, flexible schedule. You will roleplay live conversations with an AI voice model, score exchanges, and compare AI-generated images and videos to judge which follows prompts better.

You will explain what looked or sounded wrong in writing, remotely and at your own pace. At Outlier, your work helps shape how advanced AI systems reason and improve, contributing to real-world AI performance and providing