Remote Multimodal Evaluation Auditor (Contractor)

AuraOne, Inc.; United States; Remote; USD 45,000 - 65,000; Full time

Job summary

AuraOne, Inc. invites a meticulous remote reviewer to join its Multimodal Game Theory evaluation track. You will assess model outputs against a versioned rubric, label edge cases, and provide structured feedback for retraining the modeling team.

This contractor role requires clear written reasoning, attention to detail, and the ability to calibrate scores against gold-standard examples weekly. You must be authorized to work from the United States and able to commit at least 10 hours per week.

Qualifications

• Experience in evaluation or annotation of multimodal content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning naming the issue and rubric clause.
• Attention to detail and ability to flag prompt problems.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against rubric and assign severity tags.
• Compare paired responses and justify the stronger answer.
• Tag unsafe content with proper policy category.
• Document recurring failure modes for rubric updates.
• Calibrate reviewer scores against gold-standard examples weekly.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Multimodal Game Theory evaluation

Job description

AuraOne, Inc. invites a meticulous remote reviewer to join its Multimodal Game Theory evaluation track. You will assess model outputs against a versioned rubric, label edge cases, and provide structured feedback for retraining the modeling team.

This contractor role requires clear written reasoning, attention to detail, and the ability to calibrate scores against gold-standard examples weekly. You must be authorized to work from the United States and able to commit at least 10 hours per week.