Remote Generalist Writer — Model Evaluation & Feedback

AI Trainer Jobs; United States; Remote; USD 28,000 - 55,000; Full time

Job summary

AuraOne is seeking a remote Generalist Writer evaluator to review and label generalist prompt responses using a versioned rubric. You will compare outputs, assign severity tags, and provide structured feedback to help retrain the model.

This role emphasizes precise reasoning and consistent rubric application across tasks. Ideal candidates have prior evaluation experience, strong attention to detail, and the ability to articulate clear rationales for rubric-based judgments.

Qualifications

• Experience evaluating or annotating content in generalist prompts or related fields.
• Ability to apply complex rubrics consistently across long batches.
• Clear, structured written reasoning that identifies issues and rubric clauses.
• Reliable async availability for at least 10 hours per week.
• Familiarity with inter-rater calibration and quality control processes.

Responsibilities

• Evaluate generalist writer evaluation model outputs against a versioned rubric and assign severity tags for Generalist Writer assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.

Skills

Evaluation experience

Attention to detail

Written reasoning

Availability 10h/week

Rubric interpretation

Job description

AuraOne is seeking a remote Generalist Writer evaluator to review and label generalist prompt responses using a versioned rubric. You will compare outputs, assign severity tags, and provide structured feedback to help retrain the model.

This role emphasizes precise reasoning and consistent rubric application across tasks. Ideal candidates have prior evaluation experience, strong attention to detail, and the ability to articulate clear rationales for rubric-based judgments.