Data Science Evaluation Architect (Remote, Flexible Hours)

Weekday AI; United States; Remote; USD 165,000 - 234,000; Part time

Benefits offered by this job

Fully remote

Weekly payments

Job summary

Weekday AI seeks Data Science Experts to define evaluation frameworks for evaluating AI outputs in real-world data tasks. You will assess quality by creating robust rubrics and providing structured, evidence-based feedback.

This fully remote, independent contractor role focuses on improving AI reasoning through rigorous evaluation standards and collaboration with cross-functional teams.

Qualifications

• 5+ years of professional Data Science experience in an industry setting.
• Experience in business operations or product analytics.
• Strong background in experimental design and statistics.

Responsibilities

• Design evaluation criteria for real-world data science deliverables.
• Evaluate AI-generated and human-produced work using scoring rubrics.
• Provide written justifications for evaluations and ensure technical accuracy.
• Collaborate with cross-functional teams to improve AI training quality.
• Incorporate reviewer feedback to refine evaluation methodologies.

Skills

Data Science

Experiment design

AB testing

Statistical analysis

SQL

Python

Business analytics

Communication skills

Job description

Weekday AI seeks Data Science Experts to define evaluation frameworks for evaluating AI outputs in real-world data tasks. You will assess quality by creating robust rubrics and providing structured, evidence-based feedback.

This fully remote, independent contractor role focuses on improving AI reasoning through rigorous evaluation standards and collaboration with cross-functional teams.