Cyber Harm Safeguards Analyst

Society for AI Security; San Francisco (CA); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Equity donation matching

Generous vacation and parental leave

Flexible working hours

Office space in San Francisco

Job summary

Anthropic, a San Francisco–based AI safety company, seeks an Enforcement Analyst to review content and enforce policies across our products, focusing on cyber harm and misuse prevention. You will handle flagged activity, contribute to policy design, and collaborate with engineering and policy teams to improve detection and response.

This role may involve reviewing sensitive material, weekend escalations, and staying current with threat actor tactics.

Qualifications

• Experience in cybersecurity, including knowledge of offensive techniques, exploit development, malware analysis, or vulnerability research.
• Experience performing content review, abuse investigations, or policy enforcement at volume.
• Proficiency in SQL and/or Python for data analysis and threat detection.
• Experience identifying emerging risks and communicating findings to cross-functional teams.
• Experience working with generative AI products, including writing prompts for content review and enforcement.

Responsibilities

• Review flagged content and accounts to make enforcement decisions in line with usage policies.
• Detect and mitigate misuse of AI systems to support cyberattacks, malware development, exploitation tooling, and related harm.
• Triage and escalate high-severity cases to appropriate stakeholders
• Provide feedback to policy design teams on gaps surfaced by enforcement scenarios
• Collaborate with Engineering and Data Science to surface model errors and improve precision
• Maintain high accuracy and consistency across review queues
• Stay updated on AI policy enforcement best practices and evolving cyber threats

Skills

Cybersecurity knowledge

SQL

Python

Content review

Policy enforcement

Education

Bachelor's degree or equivalent in a relevant field

Job description

Anthropic, a San Francisco–based AI safety company, seeks an Enforcement Analyst to review content and enforce policies across our products, focusing on cyber harm and misuse prevention. You will handle flagged activity, contribute to policy design, and collaborate with engineering and policy teams to improve detection and response.

This role may involve reviewing sensitive material, weekend escalations, and staying current with threat actor tactics.