Remote AI Document Reviewer & Quality Auditor

AuraOne Human Data; United States; Remote; USD 28,000 - 48,000; Full time

Benefits offered by this job

Remote work (US eligibility)

Contractor status

Job summary

AuraOne, Inc. is seeking a remote Document Reviewer (Specialist) to evaluate document annotation model outputs, label edge cases, and provide structured feedback for retraining.

This independent contractor role requires applying a versioned rubric across long batches and maintaining calibration with gold-standard examples. Responsibilities include comparing paired responses, tagging inaccuracies, and documenting recurring failure modes for model improvements.

Qualifications

• Experience evaluating document annotation or similar content.
• Ability to apply multi-page rubrics consistently across large batches.
• Clear written reasoning that identifies issues and cites rubric clauses.
• Strong attention to detail and ability to flag prompt problems.

Responsibilities

• Evaluate document annotation outputs against a rubric and assign severity tags.
• Compare paired responses and select the stronger one with a rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Document annotation

Job description

AuraOne, Inc. is seeking a remote Document Reviewer (Specialist) to evaluate document annotation model outputs, label edge cases, and provide structured feedback for retraining.

This independent contractor role requires applying a versioned rubric across long batches and maintaining calibration with gold-standard examples. Responsibilities include comparing paired responses, tagging inaccuracies, and documenting recurring failure modes for model improvements.