Remote Editorial & Media Practice Evaluator

AI Trainer Jobs; United States; Remote; USD 62,000 - 69,000; Full time

Job summary

AuraOne is seeking an Editorial and Media Practice Reviewer for a remote evaluation track. Reviewers compare model outputs against a quality rubric, label edge cases, and provide structured feedback for retraining the modeling team.

This contractor role emphasizes consistency across long batches and precise written rationales. Responsibilities include evaluating outputs, selecting stronger responses with rationale, and tagging issues like hallucinations or unsafe content while routing ambiguous

Qualifications

• Prior evaluation, annotation, or human-rater experience in editorial and media practice review.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning that names the issue and the rubric clause.
• Strong attention to detail and availability for at least 10 hours per week (asynchronous).

Responsibilities

• Evaluate editorial and media practice evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Editorial and Media Practice

Job description

AuraOne is seeking an Editorial and Media Practice Reviewer for a remote evaluation track. Reviewers compare model outputs against a quality rubric, label edge cases, and provide structured feedback for retraining the modeling team.

This contractor role emphasizes consistency across long batches and precise written rationales. Responsibilities include evaluating outputs, selecting stronger responses with rationale, and tagging issues like hallucinations or unsafe content while routing ambiguous