Remote Image & Visual Reasoning Evaluator

AuraOne; United States; On-site; USD 34,000 - 55,000; Full time

Job summary

AuraOne is seeking an Image and Visual Reasoning Evaluator to remotely review prompts and model outputs against AuraOne's quality rubric. You will compare responses, label edge cases, and provide structured rationale to help retrain and improve the evaluation system. This contractor role requires reliable weekly availability.

The work offers flexible remote arrangements and ongoing opportunities to contribute to model evaluation and improvement initiatives at AuraOne.

Qualifications

• Prior evaluation or annotation experience on image and visual reasoning tasks.
• Ability to apply multi-page rubrics consistently across large batches.
• Clear written reasoning naming the issue and rubric clause.
• Strong attention to detail and flag prompts that are problematic.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against rubric and assign severity tags.
• Compare paired responses and select the stronger with rationale.
• Label hallucinations, policy violations with structured tags.
• Capture ambiguous prompts and route to program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples weekly.
• Document recurring failure modes for future training.

Skills

Image evaluation

Annotation

Rubric-based annotation

Inter-rater calibration

Written reasoning

Job description

AuraOne is seeking an Image and Visual Reasoning Evaluator to remotely review prompts and model outputs against AuraOne's quality rubric. You will compare responses, label edge cases, and provide structured rationale to help retrain and improve the evaluation system. This contractor role requires reliable weekly availability.

The work offers flexible remote arrangements and ongoing opportunities to contribute to model evaluation and improvement initiatives at AuraOne.