AI Content Risk Policy Trainer: Violence & Safety

Jobtailor; Seattle (WA); On-site; USD 65,000 - 90,000; Full time

Job summary

Jobtailor seeks a policy evaluation professional to assess AI model responses involving violence and related content. You will determine classifications in context, distinguishing fiction, education, history, and defensive uses from real-world uplift or harm.

Responsibilities include crafting concise rationales, refining prompts, and participating in calibration discussions to align with policy language. Strong written communication and attention to detail are essential.

Qualifications

• Strong judgment about violence in fiction, the real world, or among people in distress.
• Ability to distinguish fictional, educational, historical, and defensive violence from real-world uplift or intent to harm.
• Ability to assess meaningful real-world capability in AI model responses.
• Ability to distinguish anger, frustration, or dark humor from credible threats or crisis indicators.
• Ability to write concise rationales citing policy language and conversation details.
• Ability to write and refine adversarial or borderline prompts.
• Ability to identify policy gaps, contradictions, and emerging edge cases.
• Ability to participate in calibration discussions and apply customer policy consistently.
• Accuracy and attention to detail during repetitive, feedback-heavy evaluations.
• Clear and precise written communication.
• Ability to engage carefully, responsibly, and sustainably with graphic material.

Responsibilities

• Evaluate user requests and AI model responses involving violence, weapons, threats, and dark fiction in full conversation context
• Distinguish fictional, educational, historical, and defensive violence from requests seeking real-world uplift or expressing real intent to harm
• Assess whether model responses provide meaningful real-world capability
• Distinguish anger, frustration, and dark humor from credible threats or crisis indicators
• Select defensible classifications for ambiguous cases and write concise policy-based rationales
• Write and refine adversarial or borderline prompts
• Identify policy gaps, contradictions, and emerging edge cases and raise them with project leads and policy teams
• Participate in calibration discussions and update judgments based on stronger reasoning
• Apply customer policy consistently without substituting personal beliefs
• Maintain accuracy and attention to detail across repeated evaluations involving graphic material

Skills

Policy evaluation

Strong judgment

Written rationale

Attention to detail

Crisis indicators

Adversarial prompts

Calibration discussions

Policy gaps

Job description

Jobtailor seeks a policy evaluation professional to assess AI model responses involving violence and related content. You will determine classifications in context, distinguishing fiction, education, history, and defensive uses from real-world uplift or harm.

Responsibilities include crafting concise rationales, refining prompts, and participating in calibration discussions to align with policy language. Strong written communication and attention to detail are essential.