Remote Academic Search QA Annotator

AI Trainer Jobs; United States; Remote; USD 41,000 - 83,000; Part time

Job summary

AI Trainer Jobs is seeking an Academic Search Research QA Specialist for remote evaluation work. You will assess prompts and responses against AuraOne's rubric, label edge cases, and provide structured feedback to retrain models.

This is a contractor, remote role with US-eligible status, requiring about 10 hours per week to start and strong attention to detail. Responsibilities include evaluating outputs, selecting stronger answers with rationale, tagging content, and surfacing rubric gaps for

Qualifications

• Prior evaluation, annotation, or human-rater experience in academic search QA.
• Ability to apply multi-page rubrics consistently.
• Clear written reasoning naming the issue and rubric clause.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against rubric and assign severity tags.
• Compare paired responses and justify the stronger answer.
• Label hallucinations and unsafe content with structured tags.
• Capture ambiguous prompts and route for rubric updates.

Skills

Evaluation

Annotation

Inter-rater calibration

Rubric annotation

Severity tagging

Source grounding

Web research

Browser automation

Academic search QA

Attention to detail

Tools

Browser automation

Job description

AI Trainer Jobs is seeking an Academic Search Research QA Specialist for remote evaluation work. You will assess prompts and responses against AuraOne's rubric, label edge cases, and provide structured feedback to retrain models.

This is a contractor, remote role with US-eligible status, requiring about 10 hours per week to start and strong attention to detail. Responsibilities include evaluating outputs, selecting stronger answers with rationale, tagging content, and surfacing rubric gaps for