Remote AI Evaluation Specialist

Scale AI; New Brunswick; Remote; CAD 17,000 - 29,000; Full time

Benefits offered by this job

Flexible schedule

Job summary

Scale AI, via Outlier, seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with AI voice models, rank outputs for naturalness and prompt adherence, and provide written explanations of issues observed, all remotely and on your schedule.

Ideal candidates have strong attention to detail, enjoy careful review, and can decide clearly when both options have flaws. Bachelor’s degree is a plus.

Qualifications

• Fluent English speaker with strong written communication.
• Attention to detail and ability to replay or zoom in when something seems off.
• Ability to think on your feet and discuss realistic situations naturally.
• Experience writing short, evidence-based explanations or reviews.
• Ability to make clear decisions when both options have flaws.

Responsibilities

• Hold live, scenario-based conversations in English with AI voice models and rate naturalness, usefulness, and clarity.
• Compare AI-generated images and videos and judge adherence to prompts.
• Inspect outputs closely to spot issues like distorted anatomy, flickering, or audio-video mismatches.
• Write precise explanations in English detailing observed issues and rationale.

Skills

Fluent English

Attention to detail

Conversational thinking

Evidence-based writing

Decision making

Bachelor’s degree (nice to have)

Education

Bachelor’s degree or above

Job description

Scale AI, via Outlier, seeks detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with AI voice models, rank outputs for naturalness and prompt adherence, and provide written explanations of issues observed, all remotely and on your schedule.

Ideal candidates have strong attention to detail, enjoy careful review, and can decide clearly when both options have flaws. Bachelor’s degree is a plus.