Contract Data Scientist - Remote: Define AI Criteria

YO IT Consulting; Seattle (WA); On-site; USD 46,000 - 91,000; Full time

Benefits offered by this job

Weekly payments

Fully remote

Independent contractor

Job summary

YO IT Consulting is seeking a Data Scientist contractor for a remote project that evaluates AI-driven data science work. You will define grading criteria and provide written justifications for scores rather than delivering the work yourself.

You will score both AI-generated and human samples, design experiments, and communicate findings to senior stakeholders. Strong writing and 5+ years of industry experience are essential; knowledge of AI training or evaluation is a plus.

Qualifications

• 5+ years of professional data science experience.
• Background in business operations, product, or growth data science at top-tier tech companies.
• Strong experience in experiment design and A/B testing, metric definition, and communicating findings to executives.
• Exceptional written communication and attention to detail.
• Experience with AI training/evaluation is a plus.

Responsibilities

• Design precise, task-specific grading criteria for real-world data science deliverables.
• Score AI-generated and human work samples with written justifications for every score.
• Apply consistent, evidence-based judgment to ensure scores are reproducible.
• Incorporate structured feedback from senior reviewers and iterate quickly.

Skills

Data Science

Experiment design

SQL

Python

Communication

Job description

YO IT Consulting is seeking a Data Scientist contractor for a remote project that evaluates AI-driven data science work. You will define grading criteria and provide written justifications for scores rather than delivering the work yourself.

You will score both AI-generated and human samples, design experiments, and communicate findings to senior stakeholders. Strong writing and 5+ years of industry experience are essential; knowledge of AI training or evaluation is a plus.