Data Science Evaluation Specialist — Remote

Weekday 1; United States; Remote; USD 165,000 - 234,000; Full time

Benefits offered by this job

Fully remote

Flexible scheduling

Job summary

Weekday 1 is recruiting Data Science Experts to define high-quality evaluation frameworks for AI performance on real-world data science tasks. You will assess AI outputs with structured, evidence-based feedback and help shape how complex analytical work is judged.

Ideal candidates bring 5+ years in industry data science, strong stats, SQL and Python, and exceptional written communication to produce reproducible, defensible assessments and actionable business recommendations.

Qualifications

• 5+ years of professional experience in Data Science in an industry environment.
• Strong background in business operations, product analytics, or growth analytics at technology-driven organizations.
• Deep expertise in experimental design and A/B testing, statistical analysis, SQL and Python, metric development and measurement.
• Excellent written communication skills with the ability to explain technical reasoning clearly.
• Exceptional attention to detail and ability to evaluate work against structured rubrics.
• Comfortable receiving feedback and calibrating evaluation standards with peers.

Responsibilities

• Design detailed evaluation criteria for data science deliverables, including analyses, models, dashboards, experiment results and business insights.
• Evaluate AI-generated and human-produced work using objective scoring frameworks.
• Provide detailed written justifications for each evaluation, ensuring consistency and accuracy.
• Apply analytical judgment to produce reproducible and defensible assessments.
• Incorporate structured reviewer feedback and refine methodologies.
• Collaborate with cross-functional teams to improve AI training quality and evaluation standards.

Skills

SQL

Python

Experiment design

A/B testing

Statistical analysis

Written communication

Tools

SQL databases

Python libraries

Job description

Weekday 1 is recruiting Data Science Experts to define high-quality evaluation frameworks for AI performance on real-world data science tasks. You will assess AI outputs with structured, evidence-based feedback and help shape how complex analytical work is judged.

Ideal candidates bring 5+ years in industry data science, strong stats, SQL and Python, and exceptional written communication to produce reproducible, defensible assessments and actionable business recommendations.