Remote Visual Counting Evaluator & Rubric Annotator

AI Trainer Jobs; United States; Remote; USD 30,000 - 50,000; Part time

Job summary

AuraOne is seeking a Visual Counting Evaluator to work remotely as an independent contractor. You will review prompts and responses, compare outputs, and contribute structured feedback to help retrain the model.

You will label edge cases, annotate rubric clauses, and monitor quality with a weekly calibration workflow. Reliable, detail-oriented work for at least 10 hours a week is expected.

Qualifications

• Prior evaluation, annotation, or human-rater experience on visual counting evaluation or related content.
• Ability to apply multi-page rubrics consistently across long batches.

Responsibilities

• Evaluate model outputs against the rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following issues, and unsafe content.
• Capture ambiguous prompts and flag for rubric updates.
• Maintain reviewer quality via calibration with gold-standard examples.

Skills

Model evaluation

Rubric annotation

Severity tagging

Inter-rater calibration

Visual counting

Job description

AuraOne is seeking a Visual Counting Evaluator to work remotely as an independent contractor. You will review prompts and responses, compare outputs, and contribute structured feedback to help retrain the model.

You will label edge cases, annotate rubric clauses, and monitor quality with a weekly calibration workflow. Reliable, detail-oriented work for at least 10 hours a week is expected.