Safeguards Enforcement Lead (Cyber Harms)

Anthropic; Ashington; On-site; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Health insurance

Dental and vision insurance

Fertility benefits

Parental leave 22 weeks

Flexible paid time off

Mental health support

Equity packages

Relocation support

Home office stipend

Commuter benefits

Education stipend

Daily meals in office

Job summary

Anthropic is seeking an Enforcement Lead to manage and execute enforcement actions across its products, focusing on detecting misuse of AI systems for cyber operations. You will lead a team of Cyber Enforcement Analysts and contractors, develop enforcement frameworks, and collaborate with Policy, Engineering, and Legal teams to address high-severity cases.

You will work to uphold safety policies enabling users to interact with Anthropic's AI safely and responsibly, including weekend/holiday

Qualifications

• Proficiency in SQL and Python for data analysis and threat detection.
• Experience as a people manager.
• Experience identifying emerging risks and communicating findings to stakeholders.
• Experience performing content review, abuse investigations, or policy enforcement at volume.
• Experience working with generative AI products and prompting for content review.

Responsibilities

• Manage and execute enforcement actions across Anthropic's products and services.
• Develop strategic enforcement frameworks for flagged cyber activity.
• Lead a team of Cyber Enforcement Analysts and contractors.
• Collaborate with Safeguards Policy Design, Engineering, and Data Science teams.
• Stay current on AI policy enforcement best practices and cyber threat trends.
• Respond to escalations during weekends and holidays.

Skills

SQL

Python

People management

Risk identification

Content review

Generative AI

Cybersecurity

Policy scaling

Government liaison

Trust & safety

LLMs

Abuse monitoring

Job description

• As an Enforcement Lead, you will be responsible for managing and executing enforcement actions across our products and services, with a focus on detecting and mitigating attempts to misuse Anthropic's AI systems for malicious cyber operations
• Your work will center on developing strategic enforcement frameworks for flagged activity related to cyberattacks, malware development, and offensive exploitation
• Additionally, you will manage a team of Cyber Enforcement Analysts and contractors implementing this enforcement strategy
• Safety is core to our mission, and you'll help uphold policy enforcement so that our users can safely interact with and build on top of our products in a harmless, helpful, and honest way
• Important context for this role: In this position you may be exposed to and engage with explicit content spanning a range of topics, including those of a violent, technical, or psychologically disturbing nature
• This role may require responding to escalations during weekends and holidays
• Manage a team of Cyber Enforcement Analysts and contractors, overseeing the vision of Cyber Enforcement strategy
• Create strategies to detect and mitigate potential misuse of AI systems to facilitate cyberattacks, malware creation, exploitation tooling, and related harmful cyber operations
• Collaborate with stakeholders regarding novel, ambiguous, or high-severity cases
• Collaborate with the Safeguards Policy Design Team on policy gaps surfaced through real enforcement scenarios
• Partner with Engineering and Data Science teams to ensure tooling and measurement support enforcement operations
• Keep up to date with emerging AI policy enforcement best practices, threat actor tactics, and the evolving cyber threat landscape, using these to inform enforcement decisions

Benefits

• Comprehensive health, dental, and vision insurance for you and your dependents
• Inclusive fertility benefits via Carrot Fertility
• 22 weeks of paid parental leave
• Flexible paid time off and absence policies
• Mental health support for you and your dependents
• Competitive salary and equity packages
• Optional equity donation matching at a 1:1 ratio, up to 25% of your equity grant
• Retirement plans with competitive matching
• Life and income protection plans
• $500/month flexible wellness and time saver stipend
• Commuter benefits
• Annual education stipend
• Home office stipends
• Relocation support for those moving for Anthropic
• Daily meals and snacks in the office

• Proficiency in SQL and/or Python for data analysis and threat detection
• Experience as a people manager
• Experience identifying emerging risks and communicating findings to a diverse set of stakeholders, such as Product, Policy, Engineering, and Legal teams
• Experience performing content review, abuse investigations, or policy enforcement at volume
• Experience working with generative AI products, including writing effective prompts for content review and enforcement
• Experience in cybersecurity, including knowledge of offensive techniques, exploit development, malware analysis, or vulnerability research
• We encourage you to apply even if you do not believe you meet every single qualification
• Understanding of the challenges involved in implementing product policies at scale, including in the content moderation space
• Experience working with government agencies, regulated environments, or information sharing communities
• Experience in trust & safety, abuse investigations, cybersecurity investigations, or threat intelligence in a technology or AI company
• Experience with large language models and an understanding of how AI technology could be misused for cyber operations
• Experience operating within abuse monitoring programs or enforcement review systems