Remote AI Data Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 28,000 - 48,000; Part time

Job summary

AuraOne seeks a remote Business Owner - Video Data Collection Partner to review prompts and responses against the AuraOne quality rubric. You will compare paired outputs, label edge cases, and generate structured feedback for retraining the modeling team.

As an independent contractor, you will evaluate model outputs, tag severity, and contribute to calibration cycles. Reliable async availability for at least 10 hours per week is expected, with hourly compensation confirmed after interview.

Qualifications

• Prior evaluation, annotation, or human-rater experience on related content.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and rubric clause.
• Strong attention to detail and ability to flag foundational prompts.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to program team for rubric updates.

Skills

Inter-rater calibration

Annotation experience

Model output evaluation

Rubric-based tagging

Job description

AuraOne seeks a remote Business Owner - Video Data Collection Partner to review prompts and responses against the AuraOne quality rubric. You will compare paired outputs, label edge cases, and generate structured feedback for retraining the modeling team.

As an independent contractor, you will evaluate model outputs, tag severity, and contribute to calibration cycles. Reliable async availability for at least 10 hours per week is expected, with hourly compensation confirmed after interview.