Remote AI Evaluation Data Analyst (Contractor)

AuraOne, Inc.; United States; Remote; USD 28,000 - 48,000; Part time

Job summary

AuraOne, Inc. is seeking a Data Analyst (AI Evaluation) contractor to review AI outputs against data analyst workflows, playbooks, and policy. You will grade workflow, tone, and stakeholder fit on a structured rubric to ensure quality and compliance.

You will flag escalation issues, document the recommended next steps for the modeling team, and adjudicate rubric with care. The role is remote for US-based contractors and requires at least 10 hours per week of async availability.

Qualifications

• Direct working experience in data analyst specialist operations on real teams for Data Analyst (AI Evaluation) work.
• Clear written reasoning that names the policy or workflow being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.

Responsibilities

• Review AI outputs against current data analyst specialist operations workflows, playbooks, and firm policy for Data Analyst (AI Evaluation) assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy‑adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.
• Adjudicate disputed treatments against published playbooks or firm guidance.
• Maintain reviewer-quality scores in inter‑rater calibration cycles.

Skills

Operational review

Policy adherence

Workflow judgment

Stakeholder communication

Data Analyst specialist operations

AI model evaluation

Job description

AuraOne, Inc. is seeking a Data Analyst (AI Evaluation) contractor to review AI outputs against data analyst workflows, playbooks, and policy. You will grade workflow, tone, and stakeholder fit on a structured rubric to ensure quality and compliance.

You will flag escalation issues, document the recommended next steps for the modeling team, and adjudicate rubric with care. The role is remote for US-based contractors and requires at least 10 hours per week of async availability.