Remote Retail Shelf Vision QA Evaluation Analyst

AuraOne; United States; On-site; USD 34,440 - 55,104; Full time

Job summary

AuraOne is seeking a Remote Retail Shelf Vision Failure Analyst contractor to evaluate model outputs, label content, and provide structured feedback for model retraining. The role requires careful rubric-based annotation and ongoing calibration with the team.

Responsibilities include comparing outputs, tagging issues, and documenting recurring failure modes to improve future training. This is a remote contractor role with hourly compensation and US eligibility.

Qualifications

• Prior evaluation, annotation, or human-rater experience on similar evaluation tasks.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Strong attention to detail and the ability to flag when a prompt is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples weekly.
• Document recurring failure modes so the modeling team can target them in the next training run.

Skills

Evaluation experience

Annotation

Attention to detail

Async availability

Job description

AuraOne is seeking a Remote Retail Shelf Vision Failure Analyst contractor to evaluate model outputs, label content, and provide structured feedback for model retraining. The role requires careful rubric-based annotation and ongoing calibration with the team.

Responsibilities include comparing outputs, tagging issues, and documenting recurring failure modes to improve future training. This is a remote contractor role with hourly compensation and US eligibility.