Data Scientist (AI Trainer)

MissionHires

United States

Remote

USD 55,000 - 172,000

Full time

17 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

MissionHires is seeking a remote, contract-based Data Science Task Designer and Evaluator. The role focuses on creating realistic data-science tasks, anonymizing materials, and evaluating frontier AI outputs against professional standards.

You will develop Grading Rubrics, identify failures like leakage or confounding, and collaboratively refine tasks from real-world workflows. Strong English, 10+ hours/week, and US/Canada/UK residency are expected.

Qualifications

  • 2+ years of professional applied data science experience preferred in analytics, ML, statistics, experimentation, or data engineering; substantial practical work may count.
  • Bachelor’s degree in progress or completed in a quantitative field; advanced degrees are a plus but not required.
  • Working proficiency in Python and/or SQL, including writing, debugging, reading, and explaining analysis code.
  • Depth in at least one area: product/business analytics, applied ML, statistics and experimentation, forecasting, data engineering, or NLP/computer vision with shipped work.
  • Experience owning a multi-step analysis from raw data through cleaning, modeling, interpretation, and decision-making.
  • Ability to evaluate validity of results and explain issues like leakage, confounding, selection effects, and multiple testing.
  • Full professional or native-level English, with clear explanations of complex reasoning and why outputs may be incorrect.
  • Availability for at least 10 hours per week; fully remote contract work with flexible scheduling.
  • Based in the United States, Canada, or the United Kingdom; complete unpaid skills assessment (2–4 hours) independently without AI tools.
  • Pay of $40 to $125 USD per hour, paid via PayPal on a regular cadence.
  • Remote work permanently with potential for full-time hours.
  • Multiple-day task timers enabling distribution of work across days.

Responsibilities

  • Design challenging, realistic data science tasks based on professional workflows with scenarios, prompts, and supporting materials.
  • Create or anonymize realistic professional files required to complete each task.
  • Run tasks through frontier AI models and evaluate analyses, code, models, SQL queries, and deliverables against standards.
  • Compare model outputs from identical prompts and files, determine strengths/weaknesses, and document gaps.
  • Write detailed grading rubrics and tests defining correct deliverables with data-quality handling and valid interpretations.
  • Identify and document concrete failures such as leakage, confounding, or incorrect joins and unsupported interpretations.
  • Contribute expertise across analytics, ML, experimentation, forecasting, and data-engineering-adjacent work.
  • Review and refine tasks developed by other subject-matter experts; propose and run complete task scenarios with feedback iterations.
  • Provide timely feedback, onboarding materials, and support, including office hours and Slack interaction.

Skills

Python
SQL
English proficiency
Attention to detail
Independent analysis

Education

Quantitative degree (bachelors in progress or completed)
Advanced degree a plus

Tools

Claude
ChatGPT

Job description

When AI gets the details wrong in your field, real people pay for it. The people who know the work best have to be the ones training these models. You’re not being replaced by AI. You’re making sure it’s used accurately and safely.

How You'll Contribute
  • Design challenging, realistic data science tasks based on professional workflows, including scenarios, colleague-style prompts, and supporting materials such as datasets, schemas, notebooks, metric definitions, and stakeholder briefs.
  • Create or anonymize realistic professional files needed to complete each task.
  • Run tasks through frontier AI models and evaluate the resulting analyses, code, models, SQL queries, and written deliverables against professional standards.
  • Compare model outputs produced from identical prompts and files, determine which is stronger, and document where each response falls short.
  • Write detailed grading rubrics and tests defining what a correct deliverable must include, such as appropriate data-quality handling, statistical tests, validation methods, and interpretations.
  • Identify and document concrete failures, including data leakage, confounding, incorrect joins, misapplied statistical tests, fabricated or ignored data, code that does not meet the brief, and unsupported interpretations.
  • Contribute expertise across analytics, machine learning, experimentation, forecasting, and data-engineering-adjacent work.
  • Review and refine tasks developed by other subject-matter experts.
  • Propose, build, and run complete task scenarios based on real professional experience, incorporating expert-review feedback and revisions.
What Makes You a Great Fit
  • 2+ years of professional applied data science experience preferred in analytics, machine learning, statistical modeling, experimentation, analysis-adjacent data engineering, or research; candidates with less experience may be considered if their practical work is substantial.
  • Bachelor’s degree in progress or completed in a quantitative field; advanced degrees are a plus but not required.
  • Working proficiency in Python and/or SQL, including the ability to write, debug, read, and explain analysis code.
  • Depth in at least one area: product or business analytics and BI, applied machine learning, statistics and experimentation, forecasting, analysis-adjacent data engineering, or applied NLP/computer vision with shipped work.
  • Experience independently owning a multi-step analysis from raw or messy data through cleaning, modeling, interpretation, and decision-making.
  • Ability to evaluate whether results are valid and explain issues such as leakage, confounding, selection effects, multiple testing, power, and business or scientific interpretation.
  • Current or recent hands‑on individual-contributor experience; candidates whose experience is solely managerial are not a fit.
  • Ability to draw on real‑world workflows and professional experience to create scenarios that test whether AI can perform the work correctly.
  • Full professional or native‑level written and spoken English, with the ability to clearly explain complex reasoning and why an output is incorrect.
  • Strong attention to detail, comfort with ambiguity, and the ability to verify claims against underlying data, sources, or facts.
  • Ability to independently interpret feedback, determine what is correct, and apply it without extensive direction.
  • Ability to ramp quickly on unfamiliar work using written instructions and incomplete source material.
  • General familiarity with AI and LLM tools such as Claude or ChatGPT and the judgment to distinguish sound reasoning from plausible but incorrect answers.
  • Availability for at least 10 hours per week, with consistent availability preferred.
  • Based in the United States, Canada, or the United Kingdom.
  • Applicants must complete an unpaid skills assessment lasting approximately 2 to 4 hours; all assessment work must be completed independently without AI tools.
  • Pay of $40 to $125 USD per hour, paid via PayPal on a regular cadence.
  • Fully remote contract work with flexible scheduling.
  • Minimum commitment of 10 hours per week with no weekly maximum; full‑time hours may be available.
  • Multi‑day task timers that allow work to be distributed across several days.
  • Access to Claude and ChatGPT through the project; no personal subscription is required.
  • Regular paid projects after successful onboarding and acceptance.
  • Short paid training project covering task, file, and rubric development.
  • Written feedback, onboarding materials, dedicated Slack support, and office hours.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Science AI Trainer
Data Science AI Trainer

MissionHires • United States

Remote
USD 55,000 - 172,000
Fully remote contract
Flexible scheduling
Minimum 10 hours/week
Math AI Trainer
Math AI Trainer

MissionHires • United States

Remote
USD 55,000 - 172,000
Hourly contract
Fully remote
Flexible scheduling
+5
Life Sciences AI Trainer
Life Sciences AI Trainer

MissionHires • United States

Remote
USD 55,000 - 172,000
Pay via PayPal
Fully remote contract
Flexible scheduling
+4
Data Science Expert, AI Trainer
Data Science Expert, AI Trainer

Sovrano AI • United States

Remote
USD 55,000 - 172,000
Coding AI Trainer
Coding AI Trainer

MissionHires • United States

Remote
USD 55,000 - 172,000
Fully remote contract
Flexible scheduling
Minimum 10 hours/week
+3
Accounting AI Trainer
Accounting AI Trainer

MissionHires • United States

Remote
USD 55,000 - 172,000
Remote work
Hourly pay $40–$125 USD
10+ hours per week
+2
Biology AI Trainer
Biology AI Trainer

MissionHires • United States

Remote
USD 55,000 - 172,000
Remote contract
Flexible scheduling
PayPal payments
+2
Data Scientist
Data Scientist

Barclay Simpson • United States

Remote
USD 55,000 - 172,000
Data Scientist
Data Scientist

Joinkeywork • United States

Remote
USD 55,000 - 110,000
Remote work
Contractor role
Access to future opportunities
Data Science AI Response Evaluator
Data Science AI Response Evaluator

OpenTrain AI, Inc. • United States

Remote
USD 136,000 - 194,000