Remote AI QA Specialist: Coding Agent Evaluations

AuraOne Human Data; EUA; Teletrabalho; USD 55 000 - 96 000; Tempo parcial

Vantagens oferecidas por esta oferta de emprego

Prior experience training

AI workflow tooling familiarity

Bilingual experience for cross-region

Resumo da oferta

AuraOne, Inc. is seeking a remote Member of Technical Staff for Coding Agent Research & Evaluations as an independent contractor. You will review AI outputs, assess workflow alignment with firm policy, and document the right next steps to enable model training.

You’ll apply structured rubrics to rate tone, escalation, and stakeholder fit, flag risk, and ensure quality across long batches. This is US-remote, contractor-based work with compensation discussed after interviews.

Qualificações

• Experience reviewing AI outputs against workflows and firm policy.
• Ability to apply multi-page rubrics consistently across long batches.
• Clear written reasoning that names the policy or workflow applied.

Responsabilidades

• Review AI outputs against workflows, playbooks, and policy for assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps.
• Capture the right next step so the modeling team can train on it.
• Adjudicate disputed playbook calls using firm guidance.
• Audit batches for rubric consistency and report drift.

Conhecimentos

Operational review

Policy adherence

Workflow judgment

Stakeholder communication

Descrição da oferta de emprego

AuraOne, Inc. is seeking a remote Member of Technical Staff for Coding Agent Research & Evaluations as an independent contractor. You will review AI outputs, assess workflow alignment with firm policy, and document the right next steps to enable model training.

You’ll apply structured rubrics to rate tone, escalation, and stakeholder fit, flag risk, and ensure quality across long batches. This is US-remote, contractor-based work with compensation discussed after interviews.