Remote Image Reasoning Evaluator & Rubric Auditor

AuraOne; United States; On-site; USD 40,000 - 57,000; Full time

Job summary

AuraOne is seeking a remote Image Reasoning Generalist reviewer to evaluate prompts and model outputs against our quality rubric. This contractor role emphasizes labeling edge cases, providing structured feedback, and supporting retraining efforts.

You will compare paired responses, assign severity tags, and document recurring failure modes to help the modeling team improve future iterations. Prior annotation experience and clear written reasoning are essential.

Qualifications

• Prior evaluation, annotation, or human-rater experience on image reasoning generalist evaluation or adjacent content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning naming the issue and the rubric clause.

Responsibilities

• Evaluate image reasoning generalist evaluation outputs against a rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route to the program team for rubric updates.
• Calibrate reviewer-quality scores against gold-standard examples weekly.
• Document recurring failure modes to guide the next training run.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Image Reasoning Generalist evaluation

Job description

AuraOne is seeking a remote Image Reasoning Generalist reviewer to evaluate prompts and model outputs against our quality rubric. This contractor role emphasizes labeling edge cases, providing structured feedback, and supporting retraining efforts.

You will compare paired responses, assign severity tags, and document recurring failure modes to help the modeling team improve future iterations. Prior annotation experience and clear written reasoning are essential.