Remote Data Science Quality Evaluator for AI Work

YO IT Consulting; Abu Dhabi; On-site; AED 246,000 - 424,000; Full time

Benefits offered by this job

Weekly payments via Stripe

Weekly payments via Wise

Job summary

YO IT Consulting is offering a remote contractor Data Science Evaluator role in Abu Dhabi Emirate, United Arab Emirates. The position focuses on defining quality standards for realistic data science tasks and assessing how effectively AI systems and human contributors meet them.

The evaluator will design scoring rubrics, apply criteria to analyses, models, dashboards, and written recommendations, and provide clear, reproducible justifications for every score.

Qualifications

• At least five years of professional data science experience in an industry environment.
• Experience supporting product, growth, or business-operations decisions within a leading technology company.
• Advanced capability in experiment design, A-B testing, metric development, SQL, Python, and statistics.
• Ability to communicate technical findings clearly to executive stakeholders.

Responsibilities

• Analyze realistic data science tasks and identify essential quality requirements.
• Design detailed grading criteria tailored to objective and expected outputs.
• Define scoring levels for analytical correctness, methodology, completeness, and business usefulness.
• Review AI-generated and human-produced data science work against approved criteria.
• Score analyses, models, dashboards, experiment readouts, and recommendation documents.
• Provide a clear written justification supporting every assigned score.
• Verify whether analytical methods are appropriate for the available data and business question.
• Examine metric definitions, assumptions, calculations, and interpretation for accuracy.
• Assess experiment design, sample selection, control groups, and A-B testing conclusions.
• Review SQL and Python work for technical correctness and reproducibility.
• Evaluate whether dashboards communicate relevant information without misleading users.
• Determine whether recommendations are supported by evidence and suitable for executive audiences.
• Apply standards consistently across comparable tasks and submissions.

Skills

Data science quality evaluation

Task-specific rubric design

Evidence-based scoring

AI output assessment

Statistical analysis

Experiment design

A-B testing

Metric development

SQL analysis

Python analysis

Reproducibility assessment

Written score justification

Reviewer calibration

Analytical quality assurance

Tools

SQL

Python

Job description

YO IT Consulting is offering a remote contractor Data Science Evaluator role in Abu Dhabi Emirate, United Arab Emirates. The position focuses on defining quality standards for realistic data science tasks and assessing how effectively AI systems and human contributors meet them.

The evaluator will design scoring rubrics, apply criteria to analyses, models, dashboards, and written recommendations, and provide clear, reproducible justifications for every score.