AI Evaluation Specialist - Remote, Flexible Hours

Scale AI; Dartmouth; Hybrid; CAD 24,000 - 34,000; Full time

Benefits offered by this job

Flexible schedule

Free access to leading LLMs

Global expert community

Collaborative projects

Job summary

Outlier is seeking detail-oriented English speakers to review AI-generated media, roleplay live conversations with an AI voice model, and score exchanges. You will compare images and videos, explain what looked off, and work remotely on your own schedule.

Responsibilities include evaluating image, video, and audio outputs, rating adherence and realism, and producing concise explanations in English. Flexible, freelance contractor role.

Qualifications

• Fluent English with strong written communication skills.
• Eye for detail with ability to replay, zoom, and assess outputs.
• Ability to write concise, evidence-based explanations.
• Comfort making clear decisions when both options have flaws.

Responsibilities

• Hold live, scenario-based conversations with AI voice models in English and rate naturalness and usefulness.
• Compare AI-generated images and videos and judge adherence to prompts.
• Inspect outputs closely to spot distortions, artifacts, or misalignment.
• Provide written explanations highlighting what you saw or heard.

Skills

Fluent English

Attention to detail

Clear written communication

English reasoning & explanations

Decision making under ambiguity

Education

Bachelor’s degree or above

Job description

Outlier is seeking detail-oriented English speakers to review AI-generated media, roleplay live conversations with an AI voice model, and score exchanges. You will compare images and videos, explain what looked off, and work remotely on your own schedule.

Responsibilities include evaluating image, video, and audio outputs, rating adherence and realism, and producing concise explanations in English. Flexible, freelance contractor role.