Remote AI Trainer & Evaluator - Rubric-Based Feedback

AuraOne; United States; On-site; USD 34,440 - 68,880; Part time

Benefits offered by this job

Remote work

Contractor role

Flexible hours

Job summary

AuraOne is seeking a remote AI Trainer & Evaluator to review model outputs, apply AuraOne's quality rubric, and assign severity tags. You will compare paired responses, select the stronger answer, and provide a concise rationale for retraining.

This independent contractor role offers flexible hours, US eligibility, and remote work from anywhere in the United States. You will document recurring failure modes, tag unsafe content, and help calibrate reviewer scores against gold-standard examples

Qualifications

• Prior evaluation, annotation, or human-rater experience on AI trainer evaluation or related content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning naming the issue and rubric clause.
• Strong attention to detail and ability to flag prompt problems.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate ai trainer outputs against rubric and assign severity tags.
• Compare paired responses and choose the stronger with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to program team for rubric updates.
• Calibrate reviewer scores against gold-standard examples weekly.
• Document recurring failure modes for model-team training.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

AI Trainer evaluation

AI model evaluation

Job description

AuraOne is seeking a remote AI Trainer & Evaluator to review model outputs, apply AuraOne's quality rubric, and assign severity tags. You will compare paired responses, select the stronger answer, and provide a concise rationale for retraining.

This independent contractor role offers flexible hours, US eligibility, and remote work from anywhere in the United States. You will document recurring failure modes, tag unsafe content, and help calibrate reviewer scores against gold-standard examples