AI Benchmark Architect & Red-Teaming Researcher

Turing; United States; Remote; USD 55,000 - 83,000; Full time

Job summary

Turing is seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that reveal weaknesses in SOTA AI models. You will craft precise tasks, test them against leading LLMs, and diagnose failure mechanisms in reasoning paths.

Your work will shape AI evaluation frameworks, ensuring frontier models are tested on rigorous, real-world scenarios.

Qualifications

• 1+ years in investigative research, fact-checking, benchmark task authoring, red-teaming, or technical QA review.
• Experience authoring, testing, or reviewing AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or HLE.
• Strong OSINT, web search, and source verification skills with credible sources.
• Ability to inspect model reasoning and source selection to diagnose failures.
• Clear, bulletproof benchmark prompts with primary-sourced answer keys.

Responsibilities

• Obscure factual truths through niche topic research for benchmark tasks.
• Rigorously verify facts against primary sources and document URLs.
• Draft clear, answerable benchmark tasks exposing model weaknesses.
• Execute tasks against SOTA models to confirm failure triggers.
• Trace logs and prompts to pinpoint where prompts fail and why.

Skills

Analytical thinking

OSINT skills

Red-teaming

Benchmark task authoring

Technical QA review

Education

Bachelor's degree or higher in relevant field

Tools

GAIA

BrowseComp

SimpleQA

SWE-bench

Terminal-Bench

Humanity’s Last Exam (HLE)

Job description

Turing is seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that reveal weaknesses in SOTA AI models. You will craft precise tasks, test them against leading LLMs, and diagnose failure mechanisms in reasoning paths.

Your work will shape AI evaluation frameworks, ensuring frontier models are tested on rigorous, real-world scenarios.