Remote Data & Chart Analysis SME - Quality Evaluator

AuraOne; United States; On-site; USD 34,000 - 83,000; Full time

Job summary

AuraOne is seeking a Subject Matter Expert for Data & Statistical Chart Analysis to remotely evaluate prompts and responses against AuraOne's quality rubric. You will review outputs, label edge cases, and write structured feedback that helps retrain models.

The role emphasizes careful reasoning, rubric application, and documenting failure modes to support model improvements. This is a contractor position withAsync work and compensation discussed after interviews.

Qualifications

• Experience evaluating data chart analysis against rubrics with structured feedback.
• Ability to apply rubric clauses consistently across long batches.
• Clear written reasoning naming issues and the rubric clause used.
• Strong attention to detail; flag prompts that are the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Evaluate model outputs against a versioned rubric and assign severity tags.
• Compare paired responses and justify the stronger answer with rationale.
• Label hallucinations, instruction-following failures, and unsafe content.
• Capture ambiguous prompts and route for rubric updates.
• Calibrate reviewer scores against gold-standard samples weekly.
• Document recurring failure modes for model retraining.

Skills

Evaluation experience

Rubric-based annotation

Severity tagging

Inter-rater calibration

Async availability

Job description

AuraOne is seeking a Subject Matter Expert for Data & Statistical Chart Analysis to remotely evaluate prompts and responses against AuraOne's quality rubric. You will review outputs, label edge cases, and write structured feedback that helps retrain models.

The role emphasizes careful reasoning, rubric application, and documenting failure modes to support model improvements. This is a contractor position withAsync work and compensation discussed after interviews.