QA Reviewer / Adjudicator

YPAI; United States; À distance; USD 39 000 - 55 000; Plein temps

Résumé du poste

YPAI is seeking a detail-oriented reviewer to evaluate annotation and transcription work from home in the United States or other supported countries. You will assess samples against acceptance criteria, score items, and provide clear written feedback to ensure quality of data.

Responsibilities include resolving disagreements, calibrating annotators, and reporting recurring errors to improve guidelines. Applicants should be at least 18 years old and committed to precise, consistent judgment.

Qualifications

• Must have precision and willingness to reject work that fails the written criteria.
• Structured, consistent judgement and clear written feedback.
• 18 years or older.

Responsabilités

• Review sampled batches against the written acceptance criteria and score them using the project's rating criteria.
• Resolve disagreements between annotators and record your reason with clear feedback.
• Measure how often annotators agree and help calibrate them when they drift apart.
• Report recurring errors and guideline gaps to improve instructions.

Connaissances

Precision

Clear written feedback

Description du poste

Check annotation and transcription work done by others, as paid work from home. It suits precise people who give clear written feedback and reject work that fails the criteria. Prior annotation, transcription or review experience is a plus, not a requirement.

What the work is for

Your reviews decide which work goes into a delivered dataset. Work must be reviewed before a project accepts it.

The work

• Review sampled batches against the written acceptance criteria. Score them using the project's rating criteria.
• Resolve disagreements between annotators. Record your reason and give clear feedback.
• Measure how often annotators agree, and help calibrate them where they drift apart.
• Report recurring errors and guideline gaps so the instructions improve.

What to expect

• Review and adjudication run in YPAI's own review console, next to the tool the annotators used.
• Each project has its own acceptance criteria, and batches can be annotation or transcription.

You need

• Precision, and the willingness to reject work that fails the written criteria.
• Structured, consistent judgement and clear written feedback.
• 18 years or older.

Good to know

• You judge by the written criteria, even where you would have labelled an item differently.
• Work comes project by project, and there can be gaps between projects.

Pay

$28–$40/hr. Rate shown is for the United States. This role is open in 12 countries, with local rates; pick your country on YPAI.

YPAI pays every two weeks for approved work via PayPal, Wise or bank transfer. Work is by invitation after your profile is reviewed.