Model Evaluator

YPAI; United States; Remote; USD 28,000 - 39,000; Full time

Job summary

YPAI is seeking evaluators to rate AI answers from home in the United States. You’ll compare responses to project criteria and provide concrete, brief reasons for each rating.

The job emphasizes careful reading, factual checks, and explicit flagging of unsafe content, with pay issued biweekly. Ideal candidates are detail-oriented, able to explain the reasoning behind judgments, and comfortable handling potentially offensive material as part of quality assurance.

Qualifications

• Strong reading comprehension in the language evaluated.
• Ability to explain rationale behind ratings, not just scores.
• 18 years or older.

Responsibilities

• Rate and compare AI answers against project criteria.
• Write a short, concrete reason for each rating.
• Check facts in an answer and rewrite weak ones when required.
• Explicitly flag unsafe or policy-violating output.

Skills

Reading comprehension

Analytical explanation

Attention to detail

Job description

Rate and compare AI answers and explain why, as paid work from home. It suits careful readers who can say why an answer is wrong, not only that it is.

What the work is for

Your ratings and written reasons help train AI models and test their answers.

The work

• Rate and compare AI answers against the project's criteria.
• Write a short, concrete reason for each rating. It matters as much as the rating itself.
• Check the facts in an answer. Rewrite weak answers where the task asks for it.
• Flag unsafe, false or policy-breaking output explicitly.

What to expect

• Each project has its own rating criteria and instructions for judging answers.
• You work in YPAI's own review console, where each project's rating criteria are set up.

You need

• Strong reading comprehension in the language you evaluate.
• The ability to explain why, not only what.
• 18 years or older.

Good to know

• You will read unsafe or offensive output at times. Judging it is part of the work.
• Work depends on the projects running in the language you evaluate. There can be gaps between projects.

Pay

$20–$28/hr. Rate shown is for the United States. This role is open in 12 countries, with local rates; pick your country on YPAI.

YPAI pays every two weeks for approved work via PayPal, Wise or bank transfer. Work is by invitation after your profile is reviewed.