Remote Senior Journalist Evaluator & Rubric Specialist

AuraOne; United States; Remote; USD 34,000 - 62,000; Full time

Benefits offered by this job

Remote work

Contractor role

Hourly pay after interview

Job summary

AuraOne seeks an experienced Senior Journalist / Writer for a remote evaluation track. You will review prompts and responses, compare paired outputs, label edge cases, and provide structured feedback for retraining. You’ll help turn outputs into auditable labels, rationales, and regression cases for quality assurance.

The role is a contractor, remote, with hourly pay confirmed after the interview. Reliable weekly availability and strong written reasoning are essential.

Qualifications

• Experience in evaluation or human-rater work is required.
• Ability to apply multi-page rubrics consistently.
• Clear written reasoning that names the issue and rubric clause.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger with rationale.
• Label hallucinations and unsafe content with structured tags.
• Capture ambiguous prompts and route for rubric updates.
• Maintain reviewer-quality scores by calibration against gold standards.

Skills

Evaluation

Annotation

Rationale writing

Inter-rater calibration

English writing

Job description

AuraOne seeks an experienced Senior Journalist / Writer for a remote evaluation track. You will review prompts and responses, compare paired outputs, label edge cases, and provide structured feedback for retraining. You’ll help turn outputs into auditable labels, rationales, and regression cases for quality assurance.

The role is a contractor, remote, with hourly pay confirmed after the interview. Reliable weekly availability and strong written reasoning are essential.