AI Refusal Behavior Insights Analyst

Globik AI; United States; Remote; Full time

Job summary

Globik AI is seeking AI Refusal Behavior Evaluation Specialists to assess model refusals across prompts, determining whether refusals are appropriate, over-cautious, or under-cautious. You will review sample responses, document rationale against policy guidelines, and flag patterns of systematic over-refusal or under-refusal while ensuring consistency across large evaluation batches.

A reliable internet connection and a computer are required.

Qualifications

• Review model refusals and determine if a refusal happened and why.
• Judge whether refusals are appropriate, over-cautious, or under-cautious.
• Classify refusal quality (clear, evasive, unhelpful, policy-aligned, etc.).

Responsibilities

• Review model responses to determine whether a refusal occurred and why.
• Judge whether refusals are appropriate, over-cautious, or under-cautious.
• Classify refusal quality (clear, evasive, unhelpful, policy-aligned, etc.).
• Compare refusal consistency across similar or paraphrased prompts.
• Document rationale with reference to policy guidelines.
• Flag patterns of systematic over-refusal or under-refusal.
• Maintain consistent standards across large evaluation batches.

Job description

Globik AI is seeking AI Refusal Behavior Evaluation Specialists to assess model refusals across prompts, determining whether refusals are appropriate, over-cautious, or under-cautious. You will review sample responses, document rationale against policy guidelines, and flag patterns of systematic over-refusal or under-refusal while ensuring consistency across large evaluation batches.

A reliable internet connection and a computer are required.