Data Quality AI Evaluation Analyst

OpenTrain AI, Inc.; Northern (KY); On-site; USD 41,000 - 83,000; Part time

Job summary

OpenTrain AI, Inc. is seeking a part-time contractor to review datasets and task outputs used to train AI systems, focusing on accuracy, completeness, consistency and adherence to written instructions.

You will make structured decisions across high-volume review assignments, explain reasoning behind difficult calls, and flag patterns that may point to broader data issues. This role requires clear written feedback and practical resolution paths.

Qualifications

• Data review and QA experience required.
• Strong analytical judgment and attention to detail.
• Ability to apply rubrics and document decisions.

Responsibilities

• Review datasets and task outputs against quality standards.
• Find errors, inconsistencies, and results that do not follow the instructions.
• Apply rubrics consistently across large sets of data.
• Document decisions when guidelines are incomplete or unclear.
• Escalate recurring problems for further analysis.
• Give clear written and verbal feedback with practical resolution paths.

Skills

Data review

Quality assurance

Analytical judgment

Attention to detail

Tools

Spreadsheet

SQL

Job description

The work

You will review datasets and task outputs used to train AI systems. Your work will focus on accuracy, completeness, consistency, and adherence to written instructions.

You will make structured decisions across high-volume review assignments, explain the reasoning behind difficult calls, and flag patterns that may point to broader data issues.

• Review datasets and task outputs against quality standards.
• Find errors, inconsistencies, and results that do not follow the instructions.
• Apply rubrics consistently across large sets of data.
• Document decisions when guidelines are incomplete or unclear.
• Escalate recurring problems for further analysis.
• Give clear written and verbal feedback with practical resolution paths.

What it pays and takes

This is a part-time contractor role for an analytical person who can make careful decisions and communicate clear feedback. Prior AI experience is not required, but strong data review and evaluation skills are important.

• Pay: $30 to $60 per hour.
• Time: Fewer than 20 hours per week.
• Location: Open to candidates in the United States.
• Language: Fluency in English required.
• Work level: Listed as entry level.
• Required skills: Data review, quality assurance, analytical judgment, and attention to detail.
• Required tools: Spreadsheet fluency, including sorting, filtering, and simple formulas.
• Helpful experience: SQL, data querying, data annotation, labeling, or rubric-based evaluation.

About OpenTrain and AI training work

OpenTrain is a platform where people find AI training and data-labeling work, build a profile, and apply to projects. AI training work is the human review behind modern AI systems, including checking data and evaluating model-related outputs so systems can learn from reliable examples.