AI Benchmark & Red-Teaming Specialist

Turing; India; Remote; INR 1,820,000 - 3,185,000; Part time

Job summary

Turing is seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that expose weaknesses in SOTA AI models. You will design, test, and validate tasks against leading LLMs, diagnosing failure mechanisms in model reasoning paths and search processes.

Your work will shape evaluation frameworks, ensuring frontier models are challenged with rigorous real-world scenarios and high-quality source verification.

Qualifications

• 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
• Intermediate-to-advanced web search and OSINT skills; ability to navigate academic archives, local records, or obscure databases.
• Strong analytical capability to inspect model reasoning chains, search logs, and source selection to diagnose root causes of failure.
• Ability to author clear, unambiguous benchmark prompts with bulletproof, primary-sourced answer keys.

Responsibilities

• Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple queries.
• Rigorously verify facts against primary or highly credible sources, cross-check for contradictions, and document exact source URLs.
• Draft clear, answerable benchmark tasks designed to expose structural weaknesses in model research strategies.
• Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures.
• Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed.
• Apply validated "model-breaking" research patterns across new domains and, for strong performers, review tasks authored by peer trainers.

Skills

Investigative research

OSINT

Web search

Reasoning analysis

Benchmark prompts

Education

Bachelor's degree

Job description

About Turing

Turing is based in San Francisco, California. It is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.

Role Overview

We are seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that reliably expose weaknesses in state-of-the-art (SOTA) AI models (e.g., Claude Opus 5 Max). In this role, you will discover hard-to-surface facts, build precise research tasks around them, test them directly against leading LLMs, and diagnose the exact failure mechanisms in the model’s search and reasoning paths. Your work will directly shape AI evaluation frameworks, ensuring next-generation frontier models are tested against rigorous, non-trivial real-world research scenarios.

Job Requirements

• 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
• Proven experience authoring, testing, or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE).
• Intermediate-to-advanced web search and Open Source Intelligence (OSINT) skills; ability to navigate academic archives, local records, or obscure databases.
• Strong analytical capability to inspect model reasoning chains, search logs, and source selection to diagnose root causes of failure.
• Ability to author clear, unambiguous benchmark prompts with bulletproof, primary-sourced answer keys.

Job Responsibilities

• Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries.
• Rigorously verify facts against primary or highly credible sources, cross-check for contradictory evidence, and document exact source URLs.
• Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies.
• Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures.
• Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed (distinguishing genuine model limitations from ambiguous prompts).
• Apply validated "model-breaking" research patterns across new domains and, for strong performers, review tasks authored by peer trainers.

Strong Plus (Preferred Qualifications)

• Bachelor or higher degree from relevant engineering domain is a must for this position.
• Prior experience specifically designing prompt-injection or reasoning-trap tasks that break SOTA models.
• Advanced OSINT capabilities or published investigative/academic research demonstrating deep source verification.
• Niche domain expertise in areas rich with hard-to-find records (e.g., local government filings, regional sports history, legal archives, historical patents).
• Experience serving as a lead reviewer, benchmark auditor, or quality owner on AI evaluation projects.

Offer Details

• Commitments Required: Upto 40 hours per week with 4 hours of overlap with PST.
• Engagement type: Contractor
• Engagement Length: Upto 4 weeks