Remote AI Evaluator: Judge AI Outputs & Media Quality

Outlier; San Jose (CA); Hybrid; USD 17,000 - 24,000; Full time

Benefits offered by this job

Remote work

Flexible hours

Free access to LLMs

Weekly payments

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will hold live conversations with AI voice models, score exchanges, and compare generated images and videos to judge which follows the prompt best.

You will explain in writing what looked or sounded wrong, remotely and on your own schedule. As part of Outlier’s evaluation loop, your work helps shape how advanced AI systems reason, learn, and improve, turning frontier research into AI that performs better in the

Qualifications

• Fluent English with clear written and verbal communication.
• Strong attention to detail and careful review of media outputs.
• Ability to think on your feet and explain decisions clearly.

Responsibilities

• Hold live, scenario-based conversations with AI voice models.
• Rank how natural, useful, and clear each exchange sounds.
• Compare pairs of AI-generated images and videos and judge prompt adherence.
• Inspect outputs closely to spot distortions, flickering, or misalignment with audio/video.
• Write specific, evidence-based explanations in English detailing what was observed.

Skills

Fluent English

Attention to detail

Clear written communication

Verbal communication

Adaptability

Education

Bachelor’s degree preferred

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media. You will hold live conversations with AI voice models, score exchanges, and compare generated images and videos to judge which follows the prompt best.

You will explain in writing what looked or sounded wrong, remotely and on your own schedule. As part of Outlier’s evaluation loop, your work helps shape how advanced AI systems reason, learn, and improve, turning frontier research into AI that performs better in the