Remote Open Source AI Output & Policy Evaluator

AI Trainer Jobs; United States; Remote; USD 55,000 - 110,000; Full time

Job summary

AuraOne is seeking an Open Source Issue Evaluation Specialist to join as a remote contractor. You will review AI outputs across open source issue evaluation workflows, grade workflow correctness, policy adherence, and stakeholder fit, and document the right next step so the modeling team can train on it.

The role emphasizes operational judgment, tool-use assessment, and structured rubric scoring; you will help flag risks and guide improvement of prompts and playbooks.

Qualifications

• Direct working experience in open source issue evaluation specialist operations on real teams.
• Clear written reasoning that names the policy or workflow being applied.
• Bilingual experience for cross-region operations.
• Reliable async availability for at least 10 hours per week.

Responsibilities

• Review AI outputs against current open source issue evaluation specialist operations workflows, playbooks, and firm policy for Open Source Issue Evaluation Specialist assignments.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.

Skills

Operational review

Policy adherence

Workflow judgment

Stakeholder communication

AI-assisted tooling

Code review

Software testing

Agent evaluation

Open Source Issue Evaluation

Tools

AI-assisted workflow tooling

Job description

AuraOne is seeking an Open Source Issue Evaluation Specialist to join as a remote contractor. You will review AI outputs across open source issue evaluation workflows, grade workflow correctness, policy adherence, and stakeholder fit, and document the right next step so the modeling team can train on it.

The role emphasizes operational judgment, tool-use assessment, and structured rubric scoring; you will help flag risks and guide improvement of prompts and playbooks.