Remote Rubrics QA Expert — AI Workflow Evaluator

AI Trainer Jobs; United States; Remote; USD 48,000 - 69,000; Full time

Job summary

AuraOne is seeking a Rubrics Expert contractor to review AI outputs against established rubrics and policies. You will assess tone, escalation logic, and stakeholder fit, flag risks, and document the precise next steps to guide model training.

Attention to detail and the ability to apply multi-page rubrics consistently are required, along with clear written reasoning. Remote participation with flexible hours is available.

Qualifications

• Direct working experience in rubrics specialist operations on real teams.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning naming the policy or workflow being applied.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Review AI outputs against rubrics specialist workflows, playbooks, and firm policy for Rubrics Expert assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.
• Adjudicate disputed treatments against published playbooks or firm guidance.

Skills

Rubrics expert operations

Policy adherence

Workflow judgment

Stakeholder communication

Bilingual experience

Tools

AI-assisted tooling

Job description

AuraOne is seeking a Rubrics Expert contractor to review AI outputs against established rubrics and policies. You will assess tone, escalation logic, and stakeholder fit, flag risks, and document the precise next steps to guide model training.

Attention to detail and the ability to apply multi-page rubrics consistently are required, along with clear written reasoning. Remote participation with flexible hours is available.