Remote Journalist/Writer Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 28,000 - 55,000; Full time

Job summary

AI Trainer Jobs is seeking a Journalist/Writer for a remote evaluation track. You will review prompts and responses against AuraOne’s rubric, label edge cases, and write structured feedback to retrain models.

This is a remote, contract-based role with a flexible schedule and hourly compensation to be confirmed after interview. Responsibilities include evaluating model outputs, comparing responses, tagging issues, and calibrating against gold standards.

Qualifications

• Prior evaluation or annotation experience in journalist/writer content.
• Ability to apply multi-page rubrics consistently.
• Clear written reasoning naming the issue and the rubric clause.
• Reliable remote availability for 10+ hours per week.

Responsibilities

• Evaluate journalist/writer evaluation model outputs against a versioned rubric and assign severity tags for Journalist/Writer assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.

Skills

Evaluation experience

Annotation

Written reasoning

Inter-rater calibration

Job description

AI Trainer Jobs is seeking a Journalist/Writer for a remote evaluation track. You will review prompts and responses against AuraOne’s rubric, label edge cases, and write structured feedback to retrain models.

This is a remote, contract-based role with a flexible schedule and hourly compensation to be confirmed after interview. Responsibilities include evaluating model outputs, comparing responses, tagging issues, and calibrating against gold standards.