Remote Academic Search QA Specialist — Rubric Feedback

AuraOne; United States; On-site; USD 27,552 - 55,104; Part time

Job summary

AuraOne is seeking an Academic Search Research QA Specialist for a remote contractor role focused on evaluating model outputs against a rubric and generating structured feedback for retraining. The position emphasizes accuracy, clear rationale, and effective tagging of errors.

You will compare responses, label issues, and help calibrate scoring across batches, working asynchronously for at least 10 hours per week from the United States. Applicants must be authorized to work from the US.

Qualifications

• Experience with evaluation, annotation, or human-rater tasks on academic search research QA or related content.
• Ability to apply multi-page rubrics consistently across large batches.
• Clear written reasoning that names the issue and rubric clause.
• Attention to detail and ability to flag prompt problems.

Responsibilities

• Evaluate outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Document recurring failure modes and route to the program team for rubric updates.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Academic Search Research QA evaluation

Web research

Source grounding

Browser automation

Academic

Search

Job description

AuraOne is seeking an Academic Search Research QA Specialist for a remote contractor role focused on evaluating model outputs against a rubric and generating structured feedback for retraining. The position emphasizes accuracy, clear rationale, and effective tagging of errors.

You will compare responses, label issues, and help calibrate scoring across batches, working asynchronously for at least 10 hours per week from the United States. Applicants must be authorized to work from the US.