Remote PM/PRD Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 110,000 - 165,000; Full time

Job summary

AuraOne seeks a remote contractor to evaluate product management / roadmap / PRD evaluation prompts and responses against a quality rubric. You will review frontier model outputs, label edge cases, and generate structured feedback to retrain the model.

Responsibilities include comparing outputs, tagging issues, and routing ambiguous prompts for rubric updates, with a focus on clear, written rationales and consistent rubric application.

Qualifications

• Experience in product management evaluation or similar domain.
• Ability to apply rubrics consistently across long batches.
• Clear written reasoning that names the issue and rubric clause.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate product management / roadmap / PRD evaluation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and provide a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.

Skills

Evaluation

Annotation

Attention to detail

Async availability

Education

Linguistics background

Content moderation experience

Trust & Safety review

Tools

Rubrics

Job description

AuraOne seeks a remote contractor to evaluate product management / roadmap / PRD evaluation prompts and responses against a quality rubric. You will review frontier model outputs, label edge cases, and generate structured feedback to retrain the model.

Responsibilities include comparing outputs, tagging issues, and routing ambiguous prompts for rubric updates, with a focus on clear, written rationales and consistent rubric application.