Remote Experimental Design AI QA Specialist

AI Trainer Jobs; United States; Remote; USD 55,000 - 76,000; Full time

Job summary

AI Trainer Jobs is seeking an Experimental Design AI Reviewer for a remote, independent contractor role. Reviewers assess AI outputs against workflows, policy, and stakeholder fit, and document next steps for model training.

Responsibilities include grading tone and escalation, flagging policy-adherence gaps, and ensuring timely, clear written reasoning. Part-time, hourly engagement with program-defined tasks and pacing.

Qualifications

• Direct working experience in experimental design AI reviewer operations on real teams.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning naming the policy or workflow applied.
• Attention to detail and ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Review AI outputs against current experimental design AI reviewer workflows, playbooks, and firm policy for reviewer assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.

Skills

Experimental design

AI operations

Attention to detail

Policy adherence

Rubric writing

Stakeholder communication

Async communication

Bilingual experience

Tools

AI evaluation tools

Job description

AI Trainer Jobs is seeking an Experimental Design AI Reviewer for a remote, independent contractor role. Reviewers assess AI outputs against workflows, policy, and stakeholder fit, and document next steps for model training.

Responsibilities include grading tone and escalation, flagging policy-adherence gaps, and ensuring timely, clear written reasoning. Part-time, hourly engagement with program-defined tasks and pacing.