Remote Data Scientist: AI Evaluation & Quality Grading

Mercor; Germany (OH); On-site; USD 137,760 - 206,640; Part time

Job summary

Mercor connects elite creative and technical talent with leading AI research labs. Mercor is headquartered in San Francisco and offers a remote Data Scientist Talent Network contract position with compensation ranging from $100–$150 per hour.

The role focuses on designing grading criteria for data science work, evaluating outputs, and providing transparent justification for scores. Candidates should have 1+ years in data science and strong communication skills.

Qualifications

• 1+ years of professional data science experience.
• Experience at a leading technology, research, or quantitative firm.
• Strong command of Python, SQL, statistical modeling, ML, and experimentation.
• Exceptional written communication skills.
• Detail-oriented and consistent approach to evaluating complex work.
• Comfort receiving feedback and calibrating judgment against established standards.

Responsibilities

• Design precise, task-specific grading criteria for data science deliverables, including exploratory data analyses, statistical modeling work, machine learning pipelines, and A/B test write-ups.
• Evaluate AI-generated or human-created work against established criteria to ensure quality and consistency.
• Provide detailed written justifications for evaluations and scores to maintain transparency and accountability.
• Apply consistent, evidence-based judgment to ensure assessments are reproducible and defensible.
• Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Skills

Python

SQL

Statistical modelling

Machine learning

Experimentation

Written communication

Detail-oriented

Job description

Mercor connects elite creative and technical talent with leading AI research labs. Mercor is headquartered in San Francisco and offers a remote Data Scientist Talent Network contract position with compensation ranging from $100–$150 per hour.

The role focuses on designing grading criteria for data science work, evaluating outputs, and providing transparent justification for scores. Candidates should have 1+ years in data science and strong communication skills.