Remote AI Coding Sessions Evaluator — Model QA & Policy

AuraOne; United States; Remote; USD 248,000 - 303,000; Full time

Job summary

AuraOne is seeking a remote, US-eligible contractor to review AI outputs for model evaluation. You will apply multi-page rubrics, judge workflow adherence, and ensure policy conformance across long batches. The role requires clear reasoning and reliable async availability for at least 10 hours weekly.

The reviewer will grade tone, escalation logic, and stakeholder fit, flag risks, and document next steps to guide training. This is a high-attention QA contractor position.

Qualifications

• Direct working experience in builders developers share your ai coding sessions specialist operations on real teams for Software Engineers: Share Your AI Coding Sessions for Model Evaluation work.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the policy or workflow being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Review AI outputs against current builders developers share your ai coding sessions specialist operations playbooks, workflows, and firm policy for assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.
• Adjudicate disputed treatments against published playbooks or firm guidance.
• Maintain reviewer-quality scores in inter-rater calibration cycles.

Skills

Operational review

Policy adherence

Workflow judgment

Stakeholder communication

AI coding sessions ops

Job description

AuraOne is seeking a remote, US-eligible contractor to review AI outputs for model evaluation. You will apply multi-page rubrics, judge workflow adherence, and ensure policy conformance across long batches. The role requires clear reasoning and reliable async availability for at least 10 hours weekly.

The reviewer will grade tone, escalation logic, and stakeholder fit, flag risks, and document next steps to guide training. This is a high-attention QA contractor position.