Remote AI Benchmark & Red-Teaming Specialist

AuraOne Human Data

United States

Remote

USD 55,000 - 110,000

Part time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

AuraOne Human Data is seeking an AI Benchmark & Red-Teaming Specialist for a remote contractor role in the United States. You will review AI outputs, evaluate them against our workflows and policies, assign escalation levels, and document precise next steps to aid model training.

The role emphasizes clear written reasoning, rubric consistency across batches, and strong attention to policy adherence. Remote work is allowed for US-based applicants, with hourly compensation discussed during

Qualifications

  • Direct working experience in ai benchmark red teaming specialist operations on real teams for AI Benchmark & Red-Teaming Specialist work.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the policy or workflow being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Responsibilities

  • Review AI outputs against current ai benchmark red teaming specialist operations workflows, playbooks, and firm policy for AI Benchmark & Red-Teaming Specialist assignments.
  • Grade tone, escalation logic, and stakeholder fit on a structured rubric.
  • Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
  • Capture the right next step so the modeling team can train on it.
  • Adjudicate disputed treatments against published playbooks or firm guidance.
  • Maintain reviewer-quality scores in inter-rater calibration cycles.

Skills

Operational review
Policy adherence
Workflow judgment
Stakeholder communication
AI Benchmark Red Teaming specialist
LLM
Quality Management
QA
Science

Job description

AuraOne Human Data is seeking an AI Benchmark & Red-Teaming Specialist for a remote contractor role in the United States. You will review AI outputs, evaluate them against our workflows and policies, assign escalation levels, and document precise next steps to aid model training.

The role emphasizes clear written reasoning, rubric consistency across batches, and strong attention to policy adherence. Remote work is allowed for US-based applicants, with hourly compensation discussed during

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Benchmark & Red-Teaming Specialist
AI Benchmark & Red-Teaming Specialist

AuraOne Human Data • United States

Remote
USD 55,000 - 110,000
Remote AI Benchmark & Red-Teaming Specialist
Remote AI Benchmark & Red-Teaming Specialist

AI Work Expert • San Francisco (CA)

Remote
USD 110,000 - 170,000
Remote AI Workflow QA Analyst for Computer Systems
Remote AI Workflow QA Analyst for Computer Systems

Wishalicious • United States

Remote
USD 83,000 - 165,000
Remote AI Recruiting QA Specialist
Remote AI Recruiting QA Specialist

AuraOne Human Data • United States

Remote
USD 55,000 - 96,000
Prior training experience
AI workflow tooling familiarity
Bilingual experience
Remote AI Benchmark Researcher & Red-Teaming Specialist
Remote AI Benchmark Researcher & Red-Teaming Specialist

Benture • San Francisco (CA)

Remote
USD 55,000 - 110,000
Remote AI Safety Red-Team Specialist (English)
Remote AI Safety Red-Team Specialist (English)

AuraOne Human Data • United States

Remote
USD 39,000 - 44,000
Remote AI Evaluation Lead, Data Operations
Remote AI Evaluation Lead, Data Operations

AuraOne Human Data • United States

Remote
USD 34,000 - 83,000
Remote AI Training & Development QA Reviewer
Remote AI Training & Development QA Reviewer

Auraone • United States

Remote
USD 99,000 - 135,000
Remote ML Benchmark Review Data Analyst
Remote ML Benchmark Review Data Analyst

AuraOne Human Data • United States

Remote
USD 34,000 - 83,000
Remote AI Evaluation Specialist — Military Readiness
Remote AI Evaluation Specialist — Military Readiness

AuraOne Human Data • United States

Remote
USD 55,000 - 96,000