Remote AI Evaluation & Feedback Specialist

Auraone

United States

Remote

USD 34,000 - 69,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Remote work

Job summary

AuraOne is seeking remote evaluators to review exocentric video recorder outputs and judge them against our quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model.

This contractor role emphasizes written reasoning and careful annotation to drive measurable improvements. Responsibilities include tagging unsafe content, maintaining calibration scores, and coordinating with the team on rubric updates.

Qualifications

  • Prior evaluation, annotation, or human-rater experience is required.
  • Ability to apply multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the issue and rubric clause.
  • Strong attention to detail and reliability for at least 10 hours per week.

Responsibilities

  • Evaluate outputs against rubric and assign severity tags for assignments.
  • Compare paired responses and select the stronger with a rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with tags.
  • Capture ambiguous prompts and route to program team for rubric updates.
  • Maintain reviewer-quality scores by calibration against gold-standard examples weekly.

Skills

Evaluation experience
Rubric application
Attention to detail
Written reasoning
Async availability

Job description

AuraOne is seeking remote evaluators to review exocentric video recorder outputs and judge them against our quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model.

This contractor role emphasizes written reasoning and careful annotation to drive measurable improvements. Responsibilities include tagging unsafe content, maintaining calibration scores, and coordinating with the team on rubric updates.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote AI Data Evaluator: Video Recorder Evaluation
Remote AI Data Evaluator: Video Recorder Evaluation

AuraOne Human Data • United States

Remote
USD 28,000 - 41,000
Remote AI Data Evaluator: Video Review & Rubric Feedback
Remote AI Data Evaluator: Video Review & Rubric Feedback

AuraOne, Inc. • United States

Remote
USD 34,000 - 55,000
Remote Video AI Evaluation Specialist
Remote Video AI Evaluation Specialist

Auraone • United States

Remote
USD 34,000 - 55,000
Remote AI Evaluation Rubric Reviewer & Feedback Analyst
Remote AI Evaluation Rubric Reviewer & Feedback Analyst

AuraOne Human Data • United States

Remote
USD 55,000 - 103,000
Remote AI Evaluation & Annotation Specialist
Remote AI Evaluation & Annotation Specialist

Auraone • United States

Remote
USD 28,000 - 55,000
Remote 360° Video Evaluation & Annotation Specialist
Remote 360° Video Evaluation & Annotation Specialist

Auraone • United States

Remote
USD 34,000 - 55,000
Remote AI Evaluation Reviewer — Computer-Use Agent
Remote AI Evaluation Reviewer — Computer-Use Agent

AuraOne, Inc. • United States

Remote
USD 34,000 - 62,000
Remote AI Evaluation Rubric Reviewer & Feedback Specialist
Remote AI Evaluation Rubric Reviewer & Feedback Specialist

AuraOne, Inc. • United States

Remote
USD 34,000 - 55,000
Remote Video Editing Evaluation Specialist
Remote Video Editing Evaluation Specialist

Auraone • United States

Remote
USD 57,000 - 81,000
Remote work
US-eligible
Remote AI Evaluation & Feedback Specialist
Remote AI Evaluation & Feedback Specialist

Auraone • United States

Remote
USD 34,000 - 83,000
Remote work
Contractor engagement
US-eligible