Remote AI Quality Reviewer for CS PhD Operations

AuraOne, Inc.; United States; Remote; USD 96,000 - 124,000; Full time

Job summary

AuraOne, Inc. is seeking a remote review specialist to evaluate AI outputs for computer science PhD researchers operations.

You will assess workflow correctness, policy adherence, and stakeholder fit, scoring with a structured rubric and flagging any operational risks. As a contractor, you will document the right next steps to help the modeling team train on future tasks, adjudicate disputed calls, and maintain high inter-rater calibration scores while ensuring timely collaboration with

Qualifications

• Direct working experience in computer science phd researchers specialist operations on real teams.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning that names the policy or workflow being applied.

Responsibilities

• Review AI outputs against current workflows, playbooks, and firm policy for assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.
• Adjudicate disputed playbook calls using firm guidance.
• Maintain reviewer-quality scores in inter-rater calibration cycles.

Skills

Operational review

Policy adherence

Workflow judgment

Stakeholder communication

Computer Science PhD Researchers

Job description

AuraOne, Inc. is seeking a remote review specialist to evaluate AI outputs for computer science PhD researchers operations.

You will assess workflow correctness, policy adherence, and stakeholder fit, scoring with a structured rubric and flagging any operational risks. As a contractor, you will document the right next steps to help the modeling team train on future tasks, adjudicate disputed calls, and maintain high inter-rater calibration scores while ensuring timely collaboration with