AI Refusal Behavior Evaluation Specialist

Globik AI; United States; Remote; Full time

Job summary

Globik AI is seeking AI Refusal Behavior Evaluation Specialists to assess model refusals across prompts, determining whether refusals are appropriate, over-cautious, or under-cautious. You will review sample responses, document rationale against policy guidelines, and flag patterns of systematic over-refusal or under-refusal while ensuring consistency across large evaluation batches.

A reliable internet connection and a computer are required.

Qualifications

• Review model refusals and determine if a refusal happened and why.
• Judge whether refusals are appropriate, over-cautious, or under-cautious.
• Classify refusal quality (clear, evasive, unhelpful, policy-aligned, etc.).

Responsibilities

• Review model responses to determine whether a refusal occurred and why.
• Judge whether refusals are appropriate, over-cautious, or under-cautious.
• Classify refusal quality (clear, evasive, unhelpful, policy-aligned, etc.).
• Compare refusal consistency across similar or paraphrased prompts.
• Document rationale with reference to policy guidelines.
• Flag patterns of systematic over-refusal or under-refusal.
• Maintain consistent standards across large evaluation batches.

Job description

Globik AI is seeking AI Refusal Behavior Evaluation Specialists to assess whether model refusals are appropriate, over-cautious, or under-cautious. You will review refusal and compliance patterns across sensitive prompts and provide structured judgments.

Responsibilities

• Review model responses to determine whether a refusal occurred and why.
• Judge whether refusals are appropriate, over-cautious, or under-cautious.
• Classify refusal quality (clear, evasive, unhelpful, policy-aligned, etc.).
• Compare refusal consistency across similar or paraphrased prompts.
• Document rationale with reference to policy guidelines.
• Flag patterns of systematic over-refusal or under-refusal.
• Maintain consistent standards across large evaluation batches.

• Follow the assigned refusal-evaluation rubric precisely.
• Participate in calibration exercises when required.
• Clearly document ambiguous or borderline refusal cases.
• Regular quality audits will be conducted on submitted evaluations.
• A reliable internet connection and computer are required.

Pay

$15-$20/task