Remote Evaluator for Long-Horizon Planning Reasoning

AI Trainer Jobs; United States; Remote; USD 28,000 - 55,000; Part time

Job summary

AI Trainer Jobs is seeking a remote contractor to review long horizon planning reasoning outputs. You will assess outputs against a published rubric, label edge cases, and provide structured feedback for retraining the model.

The role emphasizes careful, repeatable annotation and documentation of results. The position requires prior evaluation or annotation experience, strong written reasoning, and reliable async availability for at least 10 hours per week; compensation is hourly and determined

Qualifications

• Prior evaluation, annotation, or human-rater experience on long horizon planning reasoning evaluation or adjacent content.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning naming the issue and rubric clause applied.
• Reliable asynchronous availability for at least 10 hours per week.

Responsibilities

• Evaluate long horizon planning reasoning evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.

Job description

AI Trainer Jobs is seeking a remote contractor to review long horizon planning reasoning outputs. You will assess outputs against a published rubric, label edge cases, and provide structured feedback for retraining the model.

The role emphasizes careful, repeatable annotation and documentation of results. The position requires prior evaluation or annotation experience, strong written reasoning, and reliable async availability for at least 10 hours per week; compensation is hourly and determined