Remote AI Evaluation Researcher — Benchmark & QA Expert

Crossing Hurdles; Colombia; On-site; COP 60,000,000 - 90,000,000; Full time

Job summary

Crossing Hurdles is seeking a diligent Research/QA benchmark analyst to verify information through credible sources and design demanding tasks for testing AI models. You will test outputs, analyze search trajectories, and craft precise benchmark prompts with primary-source-backed keys.

The ideal candidate has strong OSINT skills, experience with AI benchmarks, and a track record in technical QA, red-teaming, and rigorous written communication.

Qualifications

• Experience in investigative research and fact-checking.
• Ability to author AI benchmark tasks with clarity and precision.
• Experience in red-teaming and technical QA for AI systems.
• Strong web research and OSINT skills, including academic archives and specialized databases.

Responsibilities

• Investigate hard-to-find facts and verify information through credible sources.
• Design challenging research tasks to test AI models.
• Test AI model outputs and identify failures in search and reasoning.
• Analyze model search trajectories, source selection, and reasoning failures.
• Write clear benchmark prompts and accurate, primary-source-backed answer keys.

Skills

Investigative research

Fact-checking

AI benchmark authoring

Red-teaming

Technical QA

Tools

GAIA

BrowseComp

SimpleQA

SWE-bench

Terminal-Bench

Humanity's Last Exam

Job description

Crossing Hurdles is seeking a diligent Research/QA benchmark analyst to verify information through credible sources and design demanding tasks for testing AI models. You will test outputs, analyze search trajectories, and craft precise benchmark prompts with primary-source-backed keys.

The ideal candidate has strong OSINT skills, experience with AI benchmarks, and a track record in technical QA, red-teaming, and rigorous written communication.