AI Benchmark & Red-Teaming Specialist

AI Work Expert; San Francisco (CA); Remote; USD 110.000 - 170.000; Vollzeit

Zusammenfassung

Turing is seeking an AI Benchmark & Red-Teaming Specialist to advance research quality for frontier AI systems. You will design and verify benchmark tasks, perform red-teaming, and diagnose model weaknesses using primary sources and OSINT techniques.

Based remotely in the United States, you will collaborate with global researchers to craft high-integrity prompts and evaluation methods, ensuring robust model behavior and reliable results across benchmarks.

Qualifikationen

• Bachelor or higher degree from relevant engineering domain is a must.
• Prior experience designing prompt-injection or reasoning-trap tasks that break SOTA models.
• 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
• Experience authoring or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or HLE.

Aufgaben

• Obscure Fact Discovery: Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries.
• Source Verification: Rigorously verify facts against primary or highly credible sources, cross-check for contradictory evidence, and document exact source URLs.
• Task Construction: Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies.
• Red-Teaming & Model Testing: Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures.
• Failure Diagnosis & Debugging: Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed.
• Pattern Scaling & Review: Apply validated "model-breaking" research patterns across new domains and, for strong performers, review tasks authored by peer trainers.

Kenntnisse

LLM

OSINT

Web search

Model debugging

Prompt design

Ausbildung

Bachelor or higher degree in a relevant engineering domain

Tools

GAIA

BrowseComp

SimpleQA

SWE-bench

Terminal-Bench

Humanity’s Last Exam (HLE)

Jobbeschreibung

# AI Benchmark & Red-Teaming Specialistper hourFull-time Remote Last verified 1 Nov 2026Listed on Turing turing.comApplyApplyReferral link. You pay nothing; we may earn a fee.Affiliate disclosureWork Expert (WE) is an independent publishing and referral website. We are not a recruiter, hiring manager, agent or employer, and we are not affiliated with or endorsed by Turing. Applying takes you to the platform's own website, where we may be recorded as the referring source. We may receive a referral fee at no additional cost to you. Read our full affiliate disclosure →Apply within 30 days of clicking - Turing only credits applications inside its 30-day attribution window.## About the RoleBased in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L.## What You'll Do* Obscure Fact Discovery: Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries.* Source Verification: Rigorously verify facts against primary or highly credible sources, cross-check for contradictory evidence, and document exact source URLs.* Task Construction: Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies.* Red-Teaming & Model Testing: Execute tasks against designated SOTA models to confirm they reliably trigger intended model failures.* Failure Diagnosis & Debugging: Trace model run logs, search trajectories, and intermediate outputs to pinpoint exactly where and why the model failed (distinguishing genuine model limitations from ambiguous prompts).* Pattern Scaling & Review: Apply validated \"model-breaking\" research patterns across new domains and, for strong performers, review tasks authored by peer trainers.* Bachelor or higher degree from relevant engineering domain is a must for this position.* Prior experience specifically designing prompt-injection or reasoning-trap tasks that break SOTA models.## You're a Good Fit If You* Required skills: LLM* Core Experience: 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.* Benchmark Familiarity: Proven experience authoring, testing, or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE).* Analytical & OSINT Skills: Intermediate-to-advanced web search and Open Source Intelligence (OSINT) skills; ability to navigate academic archives, local records, or obscure databases.* Model Trajectory Debugging: Strong analytical capability to inspect model reasoning chains, search logs, and source selection to diagnose root causes of failure.* Precision Writing: Ability to author clear, unambiguous benchmark prompts with bulletproof, primary-sourced answer keys.Get ready for TuringDesign a resume for Turing Prepare for the Turing interview## Role HighlightsPayPay not listedContractFull-timeLocationRemoteCategoryModel evaluationPosted3 days agoJob IDTUR-113843## Pay & PayoutIndependent contractor engagement through Turing. Apply via the link below - Turing handles vetting, onboarding and payment.