Remote AI Evaluation Specialist | Flexible Hours

Scale AI; Fredericton; Remote; CAD 29,000 - 41,000; Part time

Job summary

Outlier is seeking detail-oriented, fluent English speakers to help train AI models. This remote freelance role offers flexible hours and requires careful written and spoken communication to evaluate live conversations and multimedia outputs.

Tasks include rating naturalness, usefulness and clarity of AI voice, image and video outputs, comparing prompts, inspecting outputs for issues, and providing concise explanations in English. Prior experience with evidence-based writing is preferred.

Qualifications

• Fluent English with clear written communication.
• Pursuing or holding a Bachelor’s degree or above.
• Strong attention to detail and patience to replay or zoom in.
• Comfortable thinking on your feet and discussing real scenarios.
• Experience writing short, evidence-based explanations or reviews.
• Able to make clear decisions when both options have flaws.

Responsibilities

• Hold live, scenario-based conversations in English with AI models and rank naturalness, usefulness and clarity.
• Compare AI-generated images and videos and judge which follows the prompt better.
• Inspect outputs closely for issues like distorted motion, text errors, or audio-video misalignment.
• Rate outputs on adherence, quality and realism and provide English explanations.

Skills

Fluent English

Excellent written communication

Attention to detail

Conversational reasoning

Evidence-based writing

Decision making under ambiguity

Education

Bachelor’s degree or higher

Job description

Outlier is seeking detail-oriented, fluent English speakers to help train AI models. This remote freelance role offers flexible hours and requires careful written and spoken communication to evaluate live conversations and multimedia outputs.

Tasks include rating naturalness, usefulness and clarity of AI voice, image and video outputs, comparing prompts, inspecting outputs for issues, and providing concise explanations in English. Prior experience with evidence-based writing is preferred.