Remote Spatial Reasoning Evaluation Analyst (Contractor)

AuraOne, Inc.; United States; Remote; USD 34,000 - 55,000; Full time

Job summary

AuraOne, Inc. is seeking a Spatial Reasoning Image Failure Analyst for a remote contractor role in the United States. You will evaluate model outputs, apply rubrics, and tag edge cases to generate structured feedback for retraining.

Responsibilities include comparing responses, labeling unsafe content, and documenting recurring failures to improve future training. Requires strong attention to detail and at least 10 hours/week availability.

Qualifications

• Prior evaluation, annotation, or human-rater experience in spatial reasoning image failure analysis.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and rubric clause applied.
• Strong attention to detail and ability to flag problematic prompts.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content.
• Capture ambiguous prompts and route to the program team for rubric updates.
• Calibrate reviewer-quality scores against gold-standard examples weekly.
• Document recurring failure modes to guide future training.

Skills

Model eval

Rubric annotation

Severity tagging

Inter-rater calibration

Spatial failure eval

Computer vision

Spatial reasoning

Video QA

Spatial

Reasoning

Image analysis

Job description

AuraOne, Inc. is seeking a Spatial Reasoning Image Failure Analyst for a remote contractor role in the United States. You will evaluate model outputs, apply rubrics, and tag edge cases to generate structured feedback for retraining.

Responsibilities include comparing responses, labeling unsafe content, and documenting recurring failures to improve future training. Requires strong attention to detail and at least 10 hours/week availability.