Remote AI Evaluation Specialist

Scale AI; Dollard-des-Ormeaux; Remote; CAD 24,000 - 34,000; Full time

Benefits offered by this job

Flexible schedule

Free access to leading LLMs

Global expert community

Collaborative environment

Job summary

Outlier, an AI training platform owned by Scale AI, seeks detail-oriented English speakers for remote evaluation work. You will roleplay live conversations, rank AI outputs, and provide written explanations to improve model performance.

Responsibilities include evaluating images, videos, captions, and audio for fidelity to prompts, with a focus on naturalness, quality, and realism. Upto hourly pay and flexible schedule align with project needs.

Qualifications

• Fluent English with clear written communication.
• Attention to detail and ability to replay or zoom in when something seems off.
• Ability to think on your feet and discuss realistic scenarios clearly.
• Experience writing concise, evidence-based explanations or reviews.
• Ability to make clear decisions when both options have flaws.
• Bachelor’s degree or above preferred.

Responsibilities

• Hold live scenario-based conversations in English with AI voice models and rate naturalness and usefulness.
• Compare AI-generated images and videos, judging adherence to prompts.
• Inspect outputs closely to spot artifacts like distortion or mismatches.
• Provide written explanations pinpointing what you saw or heard.

Skills

Fluent English

Attention to detail

Think on your feet

Evidence-based writing

Clear decision-making

Bachelor’s degree or above

Education

Bachelor’s degree or above

Job description

Outlier, an AI training platform owned by Scale AI, seeks detail-oriented English speakers for remote evaluation work. You will roleplay live conversations, rank AI outputs, and provide written explanations to improve model performance.

Responsibilities include evaluating images, videos, captions, and audio for fidelity to prompts, with a focus on naturalness, quality, and realism. Upto hourly pay and flexible schedule align with project needs.