Research Evaluation Specialist

OpenTrain AI, Inc.; United States; Remote; USD 41,000 - 83,000; Part time

Job summary

OpenTrain AI, Inc. seeks contractors to create difficult, evidence-based Q&A materials for advanced AI model evaluation. You will earn $30 to $60 per hour equivalent, with fewer than 20 hours per week, on a remote, part-time basis.

Applicants must have a PhD or active candidacy and deep subject-matter expertise, with fluent written English. The role involves researching primary literature, documenting citations, and writing high-quality questions and defensible answers.

Qualifications

• Fluent written English required.
• Completed PhD, active candidacy, or equivalent research experience.
• Deep subject-matter expertise and familiarity with primary literature.
• Experience finding and comparing information from authoritative references.
• Ability to write challenging, original, methodologically sound questions.

Responsibilities

• Create high-difficulty questions and defensible answers in your area of expertise.
• Research primary literature and authoritative references, documenting citations and reasoning.
• Design questions requiring advanced reasoning or synthesis across sources.
• Test questions against AI systems and refine items to resist shortcuts.
• Write clear, precise, and unambiguous question-and-answer pairs.
• Follow reviewer feedback and project guidelines and quality standards.

Skills

Analytical thinking

Written communication

Research judgement

Independent work

Education

PhD or doctoral candidacy

Job description

Create difficult, evidence-based question-and-answer materials for advanced AI model evaluation. Earn $30 to $60 per hour equivalent, paid per task that meets the project specifications, with fewer than 20 hours per week.

$30–$60/hr

equivalent, paid per accepted task

Compensation

Contract, part-time

Engagement

Remote

Location

Oct 20, 2026

Posted

Open in 51 countries

United States + more

• Algeria
• Argentina
• Austria
• Bahrain
• Bangladesh
• Belgium
• Brazil
• Canada
• Chile
• Colombia
• Denmark
• Egypt
• France
• Germany
• Ghana
• Greece
• India
• Indonesia
• Italy
• Japan
• Jordan
• Kazakhstan
• Kenya
• Kuwait
• Kyrgyzstan
• Lebanon
• Malaysia
• Mexico
• Morocco
• Netherlands
• Nigeria
• Oman
• Pakistan
• Palestine
• Peru
• Philippines
• Portugal
• Qatar
• Saudi Arabia
• Singapore
• South Africa
• Spain
• Sri Lanka
• Switzerland
• Taiwan
• Thailand
• Tunisia
• United Arab Emirates
• United States
• Uzbekistan
• Vietnam

The Work

You will create original questions and defensible answers that test advanced AI systems. Your work will measure reasoning, research judgment, synthesis, accuracy, and the ability to distinguish strong evidence from shortcuts.

• Create high-difficulty questions and answers in your area of expertise.
• Research primary literature and authoritative references, documenting citations, reasoning, and supporting evidence.
• Design questions that require advanced reasoning, methodological judgment, or synthesis across multiple sources.
• Test questions against AI systems and strengthen items that can be answered through shortcuts or oversimplification.
• Write clear, precise, and unambiguous question-and-answer pairs.
• Apply reviewer feedback and follow project guidelines and quality standards.

What It Pays And Takes

This is independent contract work for researchers and other subject-matter specialists. Prior AI training or model evaluation experience is helpful but not required.

• $30 to $60 per hour equivalent, paid per task that meets the project specifications.
• Part-time work for fewer than 20 hours per week.
• Remote role open to candidates in the countries listed for this project across the Americas, Europe, Asia, Africa, and the Middle East.
• Fluent written English is required.
• Completed PhD, active doctoral candidacy, or equivalent research experience as a specialist, researcher, or professor.
• Deep subject-matter expertise, scholarly research experience, and familiarity with primary literature.
• Strong analytical thinking, close attention to detail, and excellent written communication.
• Experience finding and comparing information from authoritative references.
• Ability to write challenging, original, and methodologically sound questions.
• Ability to work independently and reliably in a remote setting.

About AI Training Work

AI training work uses human expertise to prepare and review examples that help artificial intelligence systems produce better results. Researchers are paid for this work because strong evaluation materials require careful reasoning, reliable evidence, and deep subject knowledge.

Requirements

• Experience: Entry level