Remote Long-Horizon Model Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 30,000 - 55,000; Full time

Benefits offered by this job

Remote work

Contractor role

Job summary

AuraOne seeks a remote, contractor-level Long-Horizon Planning Model Evaluation Specialist to review long horizon planning model evaluation prompts and responses, applying a quality rubric. You will label issues, compare outputs, and contribute to retraining through structured feedback.

Ideal candidates have prior evaluation/annotation experience, strong written reasoning, and attention to detail. This role offers remote-US eligibility and flexible, async work.

Qualifications

• Experience evaluating model outputs against rubrics or similar datasets.
• Ability to label edge cases and provide structured feedback.
• Familiarity with ethics, safety review, and content moderation concepts.

Responsibilities

• Evaluate long horizon planning model evaluation outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them for rubric updates.

Skills

Model evaluation

Rubric annotation

Severity tagging

Inter-rater calibration

Job description

AuraOne seeks a remote, contractor-level Long-Horizon Planning Model Evaluation Specialist to review long horizon planning model evaluation prompts and responses, applying a quality rubric. You will label issues, compare outputs, and contribute to retraining through structured feedback.

Ideal candidates have prior evaluation/annotation experience, strong written reasoning, and attention to detail. This role offers remote-US eligibility and flexible, async work.