Remote Web QA Evaluator - Research & Rubric Specialist

AuraOne; United States; On-site; USD 55,104 - 96,432; Part time

Job summary

AuraOne is seeking a Web Browsing Research QA Specialist to remotely evaluate model outputs against a quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model.

Ideal candidates have prior evaluation or annotation experience, strong attention to detail, and the ability to work asynchronously for at least 10 hours per week. Remote-US eligibility and contractor terms apply.

Qualifications

• Experience evaluating web browsing QA outputs against rubrics.
• Ability to apply multi-page rubrics consistently across large batches.
• Clear written reasoning naming issue and rubric clause.

Responsibilities

• Evaluate QA outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to rubric updates.
• Calibrate reviewer-quality scores against gold-standard examples weekly.
• Document recurring failure modes for model training improvements.

Skills

Model output evaluation

Rubric-based annotation

Severity tagging

Inter-rater calibration

Web Browsing Research QA evaluation

Web research

Source grounding

Browser automation

Browsing

Job description

AuraOne is seeking a Web Browsing Research QA Specialist to remotely evaluate model outputs against a quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model.

Ideal candidates have prior evaluation or annotation experience, strong attention to detail, and the ability to work asynchronously for at least 10 hours per week. Remote-US eligibility and contractor terms apply.