AI Safety & Cyber Evaluations Engineer

EngineersOfAI; San Francisco, Northern (CA, KY); Hybrid; USD 300,000 - 405,000; Full time

Job summary

Anthropic is hiring Cyber Evaluations Engineers to build and run evaluations that measure cyber-relevant capabilities and safeguard robustness in our models. You'll design new evals, run per-release robustness tests, and analyze jailbreaks and prompt bypasses to understand where our safeguards hold up.

You'll also design detection probes and help shape the overall abuse-detection architecture with policy partners. The annual compensation range for this role is listed below.

Qualifications

• Experience building or running evaluations, benchmarks, or test suites for software or ML systems.
• Hands-on cybersecurity experience (e.g., CTF participation, vulnerability research, exploit development).
• Proficiency in Python.
• Strong ability to communicate evaluation results with multiple cross-functional stakeholders.

Responsibilities

• Design and run capability, uplift, and safety evaluations to assess cyber-relevant risk in new models
• Execute per-release safeguard-robustness testing ahead of major model launches
• Analyze evaluation results and communicate findings clearly to the team and to stakeholders
• Design, prototype, and tune detection probes for cyber misuse
• Work with the cyber policy team to turn policy lines into a layered, robust abuse-detection architecture, and measure its precision and coverage over time
• Build and maintain internal tooling used to run and score evaluations
• Collaborate with policy and engineering partners to translate eval findings into safeguard improvements

Skills

Python

Evaluations/Benchmarks

Cybersecurity

Communication

Education

Bachelor's degree

Tools

AI/ML evaluation frameworks

Detection tools (Sigma/YARA)

Job description

Anthropic is hiring Cyber Evaluations Engineers to build and run evaluations that measure cyber-relevant capabilities and safeguard robustness in our models. You'll design new evals, run per-release robustness tests, and analyze jailbreaks and prompt bypasses to understand where our safeguards hold up.

You'll also design detection probes and help shape the overall abuse-detection architecture with policy partners. The annual compensation range for this role is listed below.