Remote Data Analyst for AI Evaluation

Remolity, Inc.; Northern (KY); Hybrid; USD 60,000 - 90,000; Full time

Benefits offered by this job

Special Referral

Job summary

Remolity, Inc. is seeking a Data Analyst (AI Evaluation) to support a US-based, remote data quality project. You will review datasets, assess outputs against guidelines, and provide precise feedback to improve AI model training.

The role emphasizes domain knowledge over prior AI experience, with a focus on accuracy and detail. Responsibilities include identifying inconsistencies, applying rubrics, and documenting rationale for ambiguous cases while handling large volumes of tasks.

Qualifications

• 3–5+ years in data review, data quality, quality assurance, or analytical roles.
• Proven ability to review, validate, and critique data or deliverables against explicit criteria.
• Excellent ability to follow detailed written instructions and apply evaluation rubrics precisely.
• Comfort with ambiguity and making justifiable decisions when guidelines are incomplete.
• High attention to detail and strong error-spotting across repetitive tasks.
• Strong written and verbal communication for documenting findings and providing feedback.
• Baseline fluency with spreadsheets; SQL or data querying is a plus.
• Experience with data annotation, labeling, or rubric-based evaluation is advantageous.

Responsibilities

• Review datasets and task outputs for accuracy, completeness, and guideline adherence.
• Identify and flag inconsistencies, errors, and outputs deviating from instructions.
• Apply structured rubrics and evaluation criteria across large data sets.
• Exercise judgment and document rationale in ambiguous scenarios.
• Escalate recurring data issues for further analysis, not just isolated fixes.
• Provide clear, actionable written and verbal feedback with resolution paths.
• Maintain meticulous attention to detail during high-volume data review assignments.

Skills

Data review

Data quality

Quality assurance

Analytical

Attention to detail

Communication

Spreadsheets

SQL

Data annotation

Job description

Remolity, Inc. is seeking a Data Analyst (AI Evaluation) to support a US-based, remote data quality project. You will review datasets, assess outputs against guidelines, and provide precise feedback to improve AI model training.

The role emphasizes domain knowledge over prior AI experience, with a focus on accuracy and detail. Responsibilities include identifying inconsistencies, applying rubrics, and documenting rationale for ambiguous cases while handling large volumes of tasks.