Remote Cybersecurity Benchmarks Engineer

Epoch AI; United States; Remote; USD 150,000 - 350,000; Full time

Job summary

Epoch AI is seeking a Software Engineer, Cybersecurity Benchmarks to design, build and own cybersecurity benchmarks for AI evaluation, primarily using the Inspect library, and to harden the sandboxes they run in.

You will collaborate with researchers, analysts and other engineers to run benchmarks on our infrastructure, ensure results are accurate, and contribute to public research outputs that illuminate AI cybersecurity capabilities.

Qualifications

• Strong software engineering experience with complex systems and robust, maintainable code.
• Experience sandboxing software against capable adversaries.
• Red team experience ethically hacking software.
• Familiarity with agentic AI use and coding harnesses.
• Ideas and creativity for new benchmarks and experiments.
• Motivation from Epoch AI’s mission to provide rigorous, independent AI cybersecurity insights.

Responsibilities

• Design, build and maintain cybersecurity benchmarks for AI within our evaluation infrastructure, primarily using the Inspect library.
• Pitch and prototype your own ideas for new benchmarks and experiments.
• Keep the suite running so we can evaluate new model releases quickly and painlessly.
• Sandbox and harden our evaluation environments to prevent AI escape.
• Stay current on how AI agents interact with their environments.
• Use threat modelling to inform benchmarking approaches and strategy.
• Integrate benchmarks with AI providers and collaborate with researchers to ensure data and outputs are accurate and publishable.

Skills

Software engineering

Sandboxing against adversaries

Red team experience

AI coding harnesses familiarity

Creative problem solving

Motivation to Epoch AI mission

Education

Bachelor's degree in CS or related field

Tools

Inspect library

CTF infrastructure

AI coding harnesses

Agent-based AI tooling

Job description

Epoch AI is seeking a Software Engineer, Cybersecurity Benchmarks to design, build and own cybersecurity benchmarks for AI evaluation, primarily using the Inspect library, and to harden the sandboxes they run in.

You will collaborate with researchers, analysts and other engineers to run benchmarks on our infrastructure, ensure results are accurate, and contribute to public research outputs that illuminate AI cybersecurity capabilities.