AI Evaluation Specialist — Chat & Media QA

Scale AI; Milton; Remote; CAD 24,000 - 34,000; Full time

Benefits offered by this job

Free access to leading LLMs

Global freelance community

Weekly payments

Job summary

Scale AI’s Outlier program is seeking detail-oriented English speakers to review AI-generated media. You’ll roleplay conversations with AI voice models, rank outputs, and explain what looked off, all on your own schedule.

You’ll evaluate images, videos, and captions, rating adherence and realism, and provide concise explanations in English. This is a remote, flexible contract opportunity with competitive pay.)

Qualifications

• Fluent English with strong written communication.
• Attention to detail; patience to look and listen closely.
• Comfortable thinking on your feet and talking through realistic situations.
• Experience writing short, evidence-based explanations or reviews.
• Able to make clear decisions when both options have flaws.
• Nice to have: pursuing or holding a Bachelor’s degree or above.

Responsibilities

• Holding live, scenario-based conversations in English with AI voice models and ranking naturalness, usefulness, and clarity.
• Comparing pairs of AI-generated images/videos to judge prompt adherence.
• Inspecting outputs closely (zooming in/replaying) to spot issues like distorted anatomy, flickering, or audio/video misalignments.
• Writing explanations in English that point to what was seen or heard, across dimensions like adherence and realism.

Skills

Fluent English

Attention to detail

Clear written communication

Conversational thinking

Experience writing reviews

Bachelor’s degree or above

Education

Bachelor’s degree or above

Job description

Scale AI’s Outlier program is seeking detail-oriented English speakers to review AI-generated media. You’ll roleplay conversations with AI voice models, rank outputs, and explain what looked off, all on your own schedule.

You’ll evaluate images, videos, and captions, rating adherence and realism, and provide concise explanations in English. This is a remote, flexible contract opportunity with competitive pay.)