Remote Chart Understanding QA Specialist — Rubric & Feedback

AuraOne; United States; On-site; USD 27,552 - 55,104; Part time

Job summary

AuraOne seeks a remote Chart Understanding Dataset QA Specialist contractor to evaluate model outputs, compare responses, and label issues using a versioned rubric. You will help calibrate scores, identify edge cases, and document recurring failure modes for retraining and rubric updates.

The role requires clear written reasoning, strict rubric application, and the ability to work asynchronously for at least 10 hours per week from a US-eligible location.

Qualifications

• Prior evaluation, annotation, or human-rater experience on chart understanding dataset QA evaluation or adjacent content for Chart Understanding Dataset QA Specialist work.
• Clear written reasoning that names the issue and the rubric clause being applied.
• Comfort applying multi-page rubrics consistently across long batches.

Responsibilities

• Evaluate chart understanding dataset QA evaluation model outputs against a versioned rubric and assign severity tags for assignments.
• Compare paired responses and pick the stronger answer with a written rationale.
• Label hallucinations, instruction-following failures, and unsafe content with structured tags.
• Capture ambiguous prompts and route them back to the program team for rubric updates.
• Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
• Document recurring failure modes so the modeling team can target them in the next training run.

Skills

Chart understanding

Rubric-based annotation

Severity tagging

Inter-rater calibration

Multimodal evaluation

Tools

Annotation platform experience

Job description

AuraOne seeks a remote Chart Understanding Dataset QA Specialist contractor to evaluate model outputs, compare responses, and label issues using a versioned rubric. You will help calibrate scores, identify edge cases, and document recurring failure modes for retraining and rubric updates.

The role requires clear written reasoning, strict rubric application, and the ability to work asynchronously for at least 10 hours per week from a US-eligible location.