Cyber Evaluations Engineer for Safe AI & Robustness

Anthropic; San Francisco, Washington (CA, District of Columbia); Hybrid; USD 300,000 - 405,000; Full time

Job summary

Anthropic is hiring a Cyber Evaluations Engineer in San Francisco to design and run evaluations that measure cyber-relevant capabilities and safeguard robustness in our models. You will develop probes, run per-release tests, and analyze data on jailbreaks and prompt bypasses to strengthen safeguards.

You will work with policy and engineering teams to translate findings into improvements and build internal tooling for evaluation scoring, with a focus on demonstrable security outcomes.

Qualifications

• Experience building or running evaluations, benchmarks, or test suites for software or ML systems, with results on short timelines.
• Hands-on cybersecurity experience (e.g., CTF, vulnerability research, exploit development).
• Proficiency in Python and ability to communicate results to cross-functional stakeholders.
• Active secret security clearance or eligibility to obtain one.

Responsibilities

• Design and run capability, uplift, and safety evaluations to assess cyber-relevant risk in new models
• Execute per-release safeguard-robustness testing ahead of major model launches
• Analyze evaluation results and communicate findings clearly to the team and stakeholders
• Design, prototype, and tune detection probes for cyber misuse
• Work with the cyber policy team to turn policy lines into a layered, robust abuse-detection architecture, and measure its precision and coverage over time
• Build and maintain internal tooling used to run and score evaluations
• Collaborate with policy and engineering partners to translate eval findings into safeguard improvements

Skills

Python

Cybersecurity

Evaluations/Benchmarks

Communication

Security Clearance

Education

Bachelor’s degree or equivalent

Job description

Anthropic is hiring a Cyber Evaluations Engineer in San Francisco to design and run evaluations that measure cyber-relevant capabilities and safeguard robustness in our models. You will develop probes, run per-release tests, and analyze data on jailbreaks and prompt bypasses to strengthen safeguards.

You will work with policy and engineering teams to translate findings into improvements and build internal tooling for evaluation scoring, with a focus on demonstrable security outcomes.