AI Evaluation Specialist - Remote, Flexible Hours

Outlier (by Scale AI); United States; Remote; USD 17,000 - 24,000; Full time

Benefits offered by this job

Flexible schedule

Free access to leading LLMs

Global expert network

Job summary

Outlier, an AI training platform by Scale AI, seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, rate outputs, and explain in writing what seems off, all on your own schedule.

Responsibilities include comparing AI-generated images and videos, labeling captions, and evaluating audio-visual alignment. Ideal candidates have strong written English, keen attention to detail, and experience writing concise evidence-based

Qualifications

• Fluent English with strong written communication.
• Meticulous attention to detail and ability to replay or zoom to inspect outputs.
• Ability to articulate a clear decision when both options have flaws.
• Experience writing concise, evidence-based explanations or reviews.

Responsibilities

• Hold live, scenario-based conversations in English with AI voice models and rank naturalness and usefulness.
• Compare AI-generated images and videos and judge prompt adherence.
• Inspect outputs to spot issues like distortion, flicker, text errors, or audio-video mismatch.
• Write specific explanations in English describing what was observed.

Skills

Fluent English

Attention to detail

Clear written communication

Explanations/written reviews

Decision making

Education

Bachelor's degree or above (nice to have)

Job description

Outlier, an AI training platform by Scale AI, seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, rate outputs, and explain in writing what seems off, all on your own schedule.

Responsibilities include comparing AI-generated images and videos, labeling captions, and evaluating audio-visual alignment. Ideal candidates have strong written English, keen attention to detail, and experience writing concise evidence-based