AI Evaluation Data Analyst — Remotely Review & Calibrate

AuraOne; United States; Remote; USD 97,000 - 138,000; Part time

Job summary

AuraOne is seeking a Data Analyst (AI Evaluation) to remotely evaluate AI outputs within data-analyst specialist operations workflows. You will review, grade, and document actions to guide model training and policy adherence for scalable assessments.

Responsibilities include applying multi-page rubrics, flagging issues with clear reasoning, and ensuring consistency across long batches. This contractor role offers flexible hours and remote work within the US.

Qualifications

• Direct working experience in data analyst specialist operations on real teams for Data Analyst (AI Evaluation) work.
• Clear written reasoning that names the policy or workflow being applied.
• Strong attention to detail and ability to flag when a prompt is the problem.

Responsibilities

• Review AI outputs against current data analyst specialist operations workflows, playbooks, and firm policy for Data Analyst (AI Evaluation) assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.
• Adjudicate disputed treatments against published playbooks or firm guidance.
• Maintain reviewer-quality scores in inter-rater calibration cycles.

Skills

Operational review

Policy adherence

Workflow judgment

Stakeholder communication

Data Analyst specialist operations

AI model evaluation

Job description

AuraOne is seeking a Data Analyst (AI Evaluation) to remotely evaluate AI outputs within data-analyst specialist operations workflows. You will review, grade, and document actions to guide model training and policy adherence for scalable assessments.

Responsibilities include applying multi-page rubrics, flagging issues with clear reasoning, and ensuring consistency across long batches. This contractor role offers flexible hours and remote work within the US.