Cybersecurity Evaluations Engineer (AI Safety & Robustness)

Anthropic; United States; Remote; USD 150,000 - 230,000; Full time

Job summary

Anthropic is hiring Cyber Evaluations Engineers to build and run evaluations that measure cyber-relevant capabilities and safeguard robustness in our models. You will design new evals, run per-release robustness testing, and analyze data on jailbreaks and prompt bypasses to understand safeguards performance.

You will design detection probes and shape the layered abuse-detection architecture with the policy team, partnering with engineering to translate findings into improvements across models.

Qualifications

• Experience building or running evaluations, benchmarks, or test suites for software or ML systems.
• Hands-on cybersecurity experience (e.g., CTF participation, vulnerability research, exploit development, or security research).
• Proficiency in Python.
• Strong ability to communicate evaluation results with cross-functional stakeholders.

Responsibilities

• Design and run evaluations to assess cyber-relevant risk in new models.
• Execute per-release safeguard-robustness testing before launches.
• Analyze results and clearly communicate findings to stakeholders.
• Design probes for cyber misuse and detection architecture with policy team.
• Build and maintain tooling to run and score evaluations.
• Collaborate with policy and engineering to translate findings into improvements.

Skills

Python

Cybersecurity experience

Communication

Job description

Anthropic is hiring Cyber Evaluations Engineers to build and run evaluations that measure cyber-relevant capabilities and safeguard robustness in our models. You will design new evals, run per-release robustness testing, and analyze data on jailbreaks and prompt bypasses to understand safeguards performance.

You will design detection probes and shape the layered abuse-detection architecture with the policy team, partnering with engineering to translate findings into improvements across models.