Remote AI Evaluation & Feedback Specialist

AI Trainer Jobs; United States; Remote; USD 34,000 - 69,000; Full time

Benefits offered by this job

Remote work

Job summary

AuraOne is seeking remote evaluators to review exocentric video recorder outputs and judge them against our quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model.

This contractor role emphasizes written reasoning and careful annotation to drive measurable improvements. Responsibilities include tagging unsafe content, maintaining calibration scores, and coordinating with the team on rubric updates.

Qualifications

• Prior evaluation, annotation, or human-rater experience is required.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and rubric clause.
• Strong attention to detail and reliability for at least 10 hours per week.

Responsibilities

• Evaluate outputs against rubric and assign severity tags for assignments.
• Compare paired responses and select the stronger with a rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to program team for rubric updates.
• Maintain reviewer-quality scores by calibration against gold-standard examples weekly.

Skills

Evaluation experience

Rubric application

Attention to detail

Written reasoning

Async availability

Job description

AuraOne is seeking remote evaluators to review exocentric video recorder outputs and judge them against our quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model.

This contractor role emphasizes written reasoning and careful annotation to drive measurable improvements. Responsibilities include tagging unsafe content, maintaining calibration scores, and coordinating with the team on rubric updates.