Remote Video Game Evaluator & Rubric Reviewer

AuraOne; United States; A distancia; USD 22.000 - 23.000; Jornada completa

Descripción de la vacante

AuraOne is seeking a Video Game Annotator to remotely evaluate prompts and responses against AuraOne’s quality rubric. You will compare outputs, label edge cases, and provide structured feedback to help retrain models. This contractor role offers $16–$17 per hour with flexible async hours and remote work across the US.

Responsibilities include evaluating model outputs, tagging content issues, and calibrating against rubrics weekly to sustain reviewer quality across batches.

Formación

• Prior evaluation, annotation, or human-rater experience on video game annotator evaluation or adjacent content.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsabilidades

• Evaluate video game annotator evaluation model outputs against a versioned rubric and assign severity tags for Video Game Annotator assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Conocimientos

Annotation experience

Attention to detail

Clear reasoning

Async availability

Descripción del empleo

AuraOne is seeking a Video Game Annotator to remotely evaluate prompts and responses against AuraOne’s quality rubric. You will compare outputs, label edge cases, and provide structured feedback to help retrain models. This contractor role offers $16–$17 per hour with flexible async hours and remote work across the US.

Responsibilities include evaluating model outputs, tagging content issues, and calibrating against rubrics weekly to sustain reviewer quality across batches.