Senior AI Data Quality Evaluator - Remote

AI Trainer Jobs; United States; Remote; USD 34,000 - 55,000; Part time

Job summary

AuraOne is seeking an Expert Data Quality Reviewer to remotely evaluate data prompts and responses per a versioned rubric. You will compare outputs, label edge cases, and provide structured feedback for retraining.

Role involves designing and auditing label taxonomies, helping turn evaluation results into auditable labels and regression cases for AuraOne Human Data. Strong writing and attention to detail are essential.

Qualifications

• Experience reviewing data quality evaluation prompts and outputs.
• Ability to apply rubrics consistently across long batches.
• Clear, written rationale naming issues and rubric clauses.
• High attention to detail and ability to flag prompts with issues.
• Reliable asynchronous availability for at least 10 hours per week.

Responsibilities

• Evaluate data quality outputs against a versioned rubric and assign severity tags.
• Compare paired responses and select the stronger one with rationale.
• Label hallucinations, instruction-following failures, and unsafe content with tags.
• Capture ambiguous prompts and route to program team for rubric updates.
• Calibrate reviewer quality against gold-standard examples weekly.

Skills

Prior evaluation experience

Rubric application

Written reasoning

Attention to detail

Async availability 10h/w

Job description

AuraOne is seeking an Expert Data Quality Reviewer to remotely evaluate data prompts and responses per a versioned rubric. You will compare outputs, label edge cases, and provide structured feedback for retraining.

Role involves designing and auditing label taxonomies, helping turn evaluation results into auditable labels and regression cases for AuraOne Human Data. Strong writing and attention to detail are essential.