Fact Checker (Remote)

Scale AI, Inc.

Alaska

Remote

USD 17,000 - 24,000

Part time

13 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Free access to leading LLMs
Join 900K+ Outlier Experts across 52+?

Job summary

Scale AI presents an opportunity via Outlier for detail-oriented English speakers to review AI-generated media and evaluate model outputs. You will engage in live conversations with AI voice models, rate realism and usefulness, and provide written explanations of judgments, all on your own schedule and remotely from the United States.

As a freelance evaluator, you may review images, videos, and audio outputs, rank them by adherence to prompts, and contribute to ongoing quality assessments.

Qualifications

  • Fluent English, with strong written communication.
  • Attention to detail and patience to review outputs carefully.
  • Ability to work independently and manage your own schedule.

Responsibilities

  • Hold live, scenario-based conversations in English with AI voice models and rate naturalness and usefulness.
  • Compare AI-generated images and videos to assess adherence to prompts.
  • Inspect outputs closely to spot issues like distortions, flicker, or misalignment between audio and video.
  • Provide written explanations of findings in English with concrete examples.
  • Perform image evaluations by selecting the better result between two options based on prompt adherence and quality.
  • Rank AI-generated videos against prompts or source clips and provide captions when needed.
  • Label and describe actions in first-person video tasks with precise captions.

Skills

Fluent English
Detail oriented
Strong written communication
Independent worker

Education

Bachelor's degree or above

Job description

Pay:

Up to USD $15 per hour, paid weekly

Location:

Remote - United States

Type:

Freelance / independent contractor

Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts work on a variety of projects, from generating training data to evaluating the performance of models.

About the opportunity:

We are seeking detail-oriented English speakers to review AI-generated media. Roleplay live conversations with an AI voice model and score each exchange, compare generated images and videos to judge which follows the prompt better, and explain in writing what looked or sounded wrong, all remotely and on your own schedule.

At Outlier, your expertise helps shape how advanced AI systems reason, learn, and improve. Through Outlier's evaluation and feedback loop, your work helps turn frontier research into AI that performs better in the real world.

Payment:
  • Project work: Earn up to USD $15 per hour
  • Additional incentives: On average, contributors earn an additional 7.5% through Missions Outliers version of surge pricing. The top quartile boosts their earnings by an average of 11%.
  • Payment schedule: Payments are processed weekly on Tuesdays, covering tasks completed during the previous pay period (Tuesday to Monday at midnight UTC), via PayPal, AirTM, or ACH transfer.
  • Rates vary based on expertise, skills assessment, location, project need, and other factors. For example, higher rates may be offered to PhDs. For non-core work, such as during initial project onboarding or project overtime phases, lower rates apply. Additional incentives data is based on payments made in the past six months and is updated quarterly. Certain projects offer incentive payments. Please review the payment terms for each project.
Responsibilities:
  • Holding a live, scenario-based conversation in English with AI voice models, then ranking how natural, useful, and clear each one sounded
  • Comparing pairs of AI-generated images and videos and judging which one better follows the prompt
  • Inspecting outputs closely, including zooming in and replaying, to spot problems like distorted anatomy, flickering or unnatural motion, text errors, and audio that doesn't match the video
  • Rating each output across dimensions like adherence, quality, and realism, then writing a specific explanation in English that points to what you saw or heard
  • Image evaluations: Compare two AI-generated images for the same text prompt and pick the better one, judging how well each follows the instructions, visual quality, and visible AI artifacts. Some tasks rank edited images against the original.
  • Video evaluations: Rank AI-generated videos against a text prompt, or edited videos against their source clip.
  • Video caption labeling: Label first-person videos of people or robots doing hands-on tasks. Contributors split each clip into individual actions and write precise captions, starting from an AI draft and correcting it.
  • Audio and voice evaluations: Rate AI-generated speech on naturalness, pronunciation, and how closely it matches a target voice. Some tasks require recording a short voice explanation of the choice.
  • Fluent English speaker with strong, clear written communication and comfortable with natural, everyday conversational speech
  • Strong attention to detail, with the patience to look and listen closely, and to replay or zoom in when something seems off
  • Comfortable thinking on your feet and talking through realistic situations in a natural, conversational way
  • Experience writing short, evidence-based explanations or reviews
  • Comfortable making a clear decision when both options have flaws
  • Nice to have: Pursuing or holding a Bachelor's degree or above
  • Work flexibly from anywhere, anytime, with no minimum hours. Set your own schedule, whether that's five or 40+ hours per week.
  • Free access to leading LLMs, worth $200+/month
  • Join 900K+ Outlier Experts across 52 countries, with $600M+ paid to experts
  • Collaborate with experts in your field to push the boundaries of pre-release models
What you'll need to sign up:
  • A valid ID and mobile phone from your country of residence
  • An up-to-date resume that highlights your expertise
  • A LinkedIn profile showing your educational background and work experience
Currently accepting applicants from:

Australia, Canada, Ireland, New Zealand, United Kingdom, and United States.

Please note:

This is a 1099 contract opportunity on the Outlier.ai platform. You will join Outlier as a freelancer/independent contractor, not an employee. Because this is a freelance opportunity, we do not offer internships, sponsorship, or employment. You must be authorized to work in your country of residence. If you are an international student, you may be able to sign up for Outlier if you are on a visa. You should contact your tax and/or immigration advisor with specific questions regarding your circumstances.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Entry Level AI Response Tester - English (Remote)
Entry Level AI Response Tester - English (Remote)

Outlier AI • Brentwood (MO)

Remote
USD 17,000 - 24,000
Flexible schedule
Free access to leading LLMs
Join 900K+ Outlier Experts
+1
Entry Level AI Response Tester - English (Remote)
Entry Level AI Response Tester - English (Remote)

Outlier AI • Glasgow (KY)

Remote
USD 12,000 - 21,000
Flexible schedule
Free access to leading LLMs
Large expert community
Copy Editor (Remote) at Outlier Minneapolis, MN
Copy Editor (Remote) at Outlier Minneapolis, MN

Outlier • Minneapolis (MN)

Remote
USD 17,000 - 24,000
Flexible hours
Work from anywhere
Free access to leading LLMs
+1
Instruction Following Rater - English Speaker (Remote)
Instruction Following Rater - English Speaker (Remote)

Outlier Ai • North Carolina

Remote
USD 17,000 - 24,000
Flexible schedule
Remote work
Free access to leading LLMs
+2
Work From Home AI Evaluator - English
Work From Home AI Evaluator - English

Outlier • United States

Remote
USD 17,000 - 24,000
Flexible schedule
Remote work from anywhere
Free access to leading LLMs
Text Annotator - Students & Graduates Welcome (Remote)
Text Annotator - Students & Graduates Welcome (Remote)

Outlier Ai • Sioux Falls (SD)

Remote
USD 17,000 - 24,000
Flexible schedule
Free AI tools access
Global expert network
+1
Work From Home Text Annotation Specialist
Work From Home Text Annotation Specialist

Scale AI, Inc. • United States

Remote
USD 17,000 - 24,000
Flexible schedule
Freelance/1099 contract
Free access to leading LLMs
+1
Work From Home AI Evaluator - English (Hiring Immediately)
Work From Home AI Evaluator - English (Hiring Immediately)

Outlier AI • United States

Remote
USD 17,000 - 24,000
Flexible schedule
Free access to leading LLMs
Global expert network
+1
Ai Conversation Trainer - English Speaker (Remote)
Ai Conversation Trainer - English Speaker (Remote)

Outlier Ai • Corpus Christi (TX)

Remote
USD 17,000 - 24,000
Flexible schedule
Free access to leading LLMs
Join 900K+ Outlier Experts across 52+?
Work From Home Language Quality Evaluator
Work From Home Language Quality Evaluator

Scale AI, Inc. • United States

Remote
USD 17,000 - 24,000
Flexible schedule
Free access to leading LLMs
Global collaboration with experts
+1