Senior Data Scientist, AI Evaluation & Scoring (Remote)

Visa Hunt; Germany (OH); On-site; USD 165,000 - 234,000; Full time

Job summary

Mercor is seeking a Data Science Expert for a contract position. This remote role offers compensation at $120-$170/hour. You will design precise grading criteria for data science deliverables and score AI-generated and human work with detailed justifications.

Apply your expertise in experiment design, A/B testing, and metric definition to help ensure scoring is reproducible and defensible, while incorporating feedback from senior reviewers and working independently to meet deadlines.

Qualifications

• 5+ years of professional data science experience in industry.
• Experience in business operations, product, or growth data science at top-tier tech firms.
• Deep fluency in experiment design, A/B testing, metric definition, and executive communication.
• Excellent written communication.
• Detail-oriented with reproducible judgment.

Responsibilities

• Design precise grading criteria for data science deliverables.
• Score AI-generated and human work with detailed justifications.
• Ensure scoring is reproducible and defensible.
• Incorporate feedback from senior reviewers and iterate.
• Work independently and asynchronously to meet deadlines while improving AI model performance.

Skills

Data science experience

Experiment design & A/B testing

SQL & Python analysis

Executive communication

Attention to detail

Tools

SQL

Python

Job description

Mercor is seeking a Data Science Expert for a contract position. This remote role offers compensation at $120-$170/hour. You will design precise grading criteria for data science deliverables and score AI-generated and human work with detailed justifications.

Apply your expertise in experiment design, A/B testing, and metric definition to help ensure scoring is reproducible and defensible, while incorporating feedback from senior reviewers and working independently to meet deadlines.