Remote AI Policy Evaluator — Violence & Fiction

Apply; Northern (KY); Hybrid; USD 62,000 - 76,000; Full time

Benefits offered by this job

Benefits eligible

Job summary

Handshake is seeking an AI Policy Specialist on the Violence & Fiction team to evaluate user requests and model responses within a full conversation context, distinguishing violence in fiction from real-world uplift and ensuring precise policy application.

You will read conversations, decide policy category, and explain reasoning clearly to train models. This remote role is US-based, with 8:00 AM–5:00 PM PT schedule and ongoing assignments starting 21 September 2026.

Qualifications

• Must have deep judgment in violence contexts, spanning fiction, real-world risk, or safety policy.

Responsibilities

• Evaluate user requests and AI model responses involving violence, weapons, threats, and dark fiction within the full conversation context.
• Distinguish fictional, educational, historical, and defensive violence from requests that seek real-world uplift or express real intent to harm.
• Assess whether a model’s response gives meaningful real-world capability, regardless of framing.
• Distinguish expressions of anger or frustration from credible threats or crisis indicators.
• Select the most defensible classification when a case is genuinely ambiguous and write concise rationales citing policy language and conversation details.
• Identify policy gaps and raise them with policy teams.
• Participate in calibration discussions and update judgment with stronger reasoning.
• Apply policy consistently without substituting personal beliefs.
• Maintain accuracy across repetitive evaluations.

Skills

Violence in fiction

Threat assessment

Trust and safety policy

Job description

Handshake is seeking an AI Policy Specialist on the Violence & Fiction team to evaluate user requests and model responses within a full conversation context, distinguishing violence in fiction from real-world uplift and ensuring precise policy application.

You will read conversations, decide policy category, and explain reasoning clearly to train models. This remote role is US-based, with 8:00 AM–5:00 PM PT schedule and ongoing assignments starting 21 September 2026.