AI Quality Evaluator (Remote, Flexible Hours)

Outlier; Portland (OR); Remote; USD 20,000 - 29,000; Part time

Benefits offered by this job

Flexible hours

Remote work from anywhere

Free access to leading LLMs

Additional incentives

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model and score each exchange, compare generated images and videos for prompt adherence, and explain in writing what looked or sounded wrong, all remotely and on your own schedule.

At Outlier, your work helps shape how advanced AI systems reason and improve through feedback loops, supporting frontier research into real-world AI performance.

Qualifications

• Fluent English with clear written communication.
• Excellent attention to detail; able to replay or zoom in when something is off.
• Bachelor’s degree or higher preferred; strong written/explanatory abilities.

Responsibilities

• Hold live, scenario-based conversations with AI voice models and rate naturalness, usefulness, and clarity.
• Compare AI-generated images and videos and judge adherence to prompts.
• Inspect outputs closely to spot problems like distorted anatomy, flickering, unrelated audio, or text errors.
• Write a concise, evidence-based explanation in English pointing to what you observed.

Skills

Fluent English

Attention to detail

Clear written communication

Decision making

Education

Bachelor’s degree or higher

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model and score each exchange, compare generated images and videos for prompt adherence, and explain in writing what looked or sounded wrong, all remotely and on your own schedule.

At Outlier, your work helps shape how advanced AI systems reason and improve through feedback loops, supporting frontier research into real-world AI performance.