AI Benchmark Task Writer | $10/hr Remote

Crossing Hurdles; India; On-site; INR 1,000,000 - 2,000,000; Full time

Job summary

Crossing Hurdles in India seeks a researcher to design and verify AI benchmarks with primary-source-backed tasks. You will investigate complex facts, verify sources, and craft challenging benchmarks for model evaluation.

The role requires strong OSINT/web research, analytical thinking, and clear benchmark writing. The position is office-based with opportunities for remote collaboration depending on project scope.

Qualifications

• Possess strong relevant experience in investigative research, fact-checking, AI benchmark authoring, red-teaming, or technical QA.
• Have experience using AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity s Last Exam.
• Demonstrate strong web research and OSINT skills, including navigating academic archives, local records, and specialized databases.
• Capable of strong analysis of model search trajectories and failures.
• Strong writing skills to craft clear and accurate benchmark tasks.

Responsibilities

• Investigate hard-to-find facts and verify information through credible sources.
• Design challenging research tasks to test AI models.
• Test AI model outputs and identify failures in search and reasoning.
• Analyze model search trajectories, source selection, and reasoning failures.
• Write clear benchmark prompts and accurate, primary-source-backed answer keys.

Skills

Investigative research

Fact-checking

AI benchmarks

Red-teaming

Technical QA

Tools

GAIA

BrowseComp

SimpleQA

SWE-bench

Terminal-Bench

Humanity s Last Exam

Job description

• Investigate hard-to-find facts and verify information through credible sources.
• Design challenging research tasks to test AI models.
• Test AI model outputs and identify failures in search and reasoning.
• Analyze model search trajectories, source selection, and reasoning failures.
• Write clear benchmark prompts and accurate, primary-source-backed answer keys.

Requirements

• Possess strong relevant experience in investigative research, fact-checking, AI benchmark authoring, red-teaming, or technical QA.
• Have experience using AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity s Last Exam.
• Demonstrate strong web research and OSINT skills, including navigating academic archives, local records, and specialized databases.
• Capable of strong analysis of model search trajectories and failures.
• Strong writing skills to craft clear and accurate benchmark tasks.