Senior Data Scientist: AI Evaluation & Scoring (Remote)

YO IT Consulting

Dubai

On-site

AED 246,000 - 424,000

Part time

3 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

YO IT Consulting is seeking a Data Scientist for AI Evaluation and Scoring in Dubai as a remote contractor. The role focuses on assessing AI-generated and human submissions using rigorously defined criteria and documented justification.

You will develop scoring rubrics, evaluate analyses, and ensure statistical validity with clear executive-facing explanations. Ideal candidates have 5+ years in data science, experience with AAA testing, metrics development, SQL, Python, and communicating complex

Qualifications

Experience evaluating AI or data science work.
Ability to design scoring rubrics and evaluation criteria.
Strong statistical analysis and experimental design skills.
Experience with AB testing and performance metrics.

Responsibilities

Examine AI- and human-generated submissions using defined criteria.
Develop task-specific evaluation rubrics for analyses, models, dashboards, readouts.
Define scoring levels for complete/partial/incorrect work.
Evaluate submissions against approved frameworks and provide written justifications.
Verify logic, statistics, metrics, calculations, and interpretation.
Assess if conclusions are supported by evidence and communicated clearly.
Review SQL and Python analyses for quality and correctness.
Check experiment designs, control groups, and AB testing conclusions.
Ensure dashboards and outputs are accurate and business-relevant.
Maintain records of decisions and confidentiality requirements.