Remote AI Benchmark & Red-Teaming Specialist

AI Work Expert; San Francisco (CA); Remote; USD 110,000 - 170,000; Full time

Job summary

Turing is seeking an AI Benchmark & Red-Teaming Specialist to advance research quality for frontier AI systems. You will design and verify benchmark tasks, perform red-teaming, and diagnose model weaknesses using primary sources and OSINT techniques.

Based remotely in the United States, you will collaborate with global researchers to craft high-integrity prompts and evaluation methods, ensuring robust model behavior and reliable results across benchmarks.

Qualifications

• Bachelor or higher degree from relevant engineering domain is a must.
• Prior experience designing prompt-injection or reasoning-trap tasks that break SOTA models.
• 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
• Experience authoring or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or HLE.

Responsibilities

• Obscure Fact Discovery: Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries.
• Source Verification: Rigorously verify facts against primary or highly credible sources, cross-check for contradictory evidence, and document exact source URLs.
• Task Construction: Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies.
• Red-Teaming & Model Testing: Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures.
• Failure Diagnosis & Debugging: Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed.
• Pattern Scaling & Review: Apply validated "model-breaking" research patterns across new domains and, for strong performers, review tasks authored by peer trainers.

Skills

LLM

OSINT

Web search

Model debugging

Prompt design

Education

Bachelor or higher degree in a relevant engineering domain

Tools

GAIA

BrowseComp

SimpleQA

SWE-bench

Terminal-Bench

Humanity’s Last Exam (HLE)

Job description

Turing is seeking an AI Benchmark & Red-Teaming Specialist to advance research quality for frontier AI systems. You will design and verify benchmark tasks, perform red-teaming, and diagnose model weaknesses using primary sources and OSINT techniques.

Based remotely in the United States, you will collaborate with global researchers to craft high-integrity prompts and evaluation methods, ensuring robust model behavior and reliable results across benchmarks.