Remote OCR & Multimodal Evaluation Specialist

AI Trainer Jobs; United States; Remote; USD 76,000 - 90,000; Full time

Job summary

AuraOne is seeking an OCR and Document Understanding Model Evaluator for a remote evaluation track. Reviewers assess document annotation prompts and responses against a quality rubric, labeling edge cases and providing structured feedback for retraining.

You will compare paired outputs, judge model behavior on text, images, and documents, and maintain calibration against gold standards each week. This is a contractor, remote role with US eligibility.

Qualifications

• Experience evaluating document annotations or related OCR/Document Understanding tasks.
• Ability to apply rubrics consistently across long batches.
• Strong written reasoning; clearly name issues and rubric clauses.
• Attention to detail and ability to flag problematic prompts.
• Comfort working asynchronously with at least 10 hours per week.

Responsibilities

• Evaluate document annotation model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger answer with written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them for rubric updates.
• Maintain reviewer-quality scores by regular calibration against gold-standard examples.

Job description

AuraOne is seeking an OCR and Document Understanding Model Evaluator for a remote evaluation track. Reviewers assess document annotation prompts and responses against a quality rubric, labeling edge cases and providing structured feedback for retraining.

You will compare paired outputs, judge model behavior on text, images, and documents, and maintain calibration against gold standards each week. This is a contractor, remote role with US eligibility.