AI Response Evaluator — Remote, Flexible Hours

Outlier; Ann Arbor (MI); Remote; USD 17,000 - 24,000; Full time

Benefits offered by this job

Flexible hours

Remote work

Free access to leading LLMs

Job summary

Outlier is offering a remote, flexible-hours opportunity to review AI-generated media by conducting live conversations with AI voice models and evaluating outputs against prompts. You will describe what looked or sounded off in writing to help improve AI reasoning and realism.

The role emphasizes careful observation, clear explanations, and consistent judgments across images, videos, and audio outputs. This is a freelance, 1099 contract-type opportunity; compensation starts at up to $15 USD per

Qualifications

• Fluent English with clear written communication.
• Strong attention to detail and careful listening.
• Pursuing or holding a Bachelor’s degree or higher.
• Able to think on your feet and discuss realistic scenarios.
• Experience writing brief, evidence-based explanations or reviews.
• Able to decide clearly when both options have flaws.

Responsibilities

• Holding a live, scenario-based conversation in English with AI voice models, then ranking how natural, useful, and clear each sounded.
• Comparing pairs of AI-generated images and videos and judging which follows the prompt.
• Inspecting outputs closely, including zooming and replaying, to spot problems like distorted anatomy, flickering or incorrect audio.
• Rating each output across dimensions like adherence, quality, and realism, then writing a specific English explanation of what you saw or heard.
• Image evaluations: compare two AI-generated images for the same prompt and pick the better one, judging instructions and visual quality.
• Video evaluations: rank AI-generated videos against a text prompt, or edited videos against their source clip.

Skills

Fluent English

Strong written communication

Attention to detail

Bachelor's degree or higher

Quick thinking

Evidence-based explanations

Education

Bachelor’s degree or higher

Job description

Outlier is offering a remote, flexible-hours opportunity to review AI-generated media by conducting live conversations with AI voice models and evaluating outputs against prompts. You will describe what looked or sounded off in writing to help improve AI reasoning and realism.

The role emphasizes careful observation, clear explanations, and consistent judgments across images, videos, and audio outputs. This is a freelance, 1099 contract-type opportunity; compensation starts at up to $15 USD per