AI Benchmark Task Writer | $10/hr Remote

Crossing Hurdles; Indonesia; On-site; IDR 1,080,886,000 - 1,621,329,000; Full time

Job summary

Crossing Hurdles seeks experts in investigative research to verify information from credible sources and design rigorous AI benchmark tasks. You will test model outputs, analyze search paths, and craft primary-source-backed answer keys.

The role emphasizes strong OSINT abilities, careful analysis, and precise technical writing to improve benchmark quality across multiple AI evaluation frameworks.

Qualifications

• Experience in investigative research and fact-checking for AI benchmarks.
• Experience authoring benchmarks and red-teaming exercises.
• Strong web research and OSINT skills for credible data collection.
• Ability to analyze model search trajectories and identify failures.
• Strong writing skills to craft benchmark tasks clearly and accurately.

Responsibilities

• Investigate hard-to-find facts and verify information through credible sources.
• Design challenging research tasks to test AI models.
• Test AI model outputs and identify failures in search and reasoning.
• Analyze model search trajectories, source selection, and reasoning failures.
• Write clear benchmark prompts and accurate, primary-source-backed answer keys.

Skills

Investigative research

Fact-checking

AI benchmarks

Red-teaming

Technical QA

Tools

GAIA

BrowseComp

SimpleQA

SWE-bench

Terminal-Bench

Humanity's Last Exam

Job description

• Investigate hard-to-find facts and verify information through credible sources.
• Design challenging research tasks to test AI models.
• Test AI model outputs and identify failures in search and reasoning.
• Analyze model search trajectories, source selection, and reasoning failures.
• Write clear benchmark prompts and accurate, primary-source-backed answer keys.

Requirements

• Possess strong relevant experience in investigative research, fact-checking, AI benchmark authoring, red-teaming, or technical QA.
• Have experience using AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity s Last Exam.
• Demonstrate strong web research and OSINT skills, including navigating academic archives, local records, and specialized databases.
• Capable of strong analysis of model search trajectories and failures.
• Strong writing skills to craft clear and accurate benchmark tasks.