AI Security Research Engineer: Evaluation & Defense

General Analysis; San Francisco, Northern (CA, KY); Hybrid; USD 150,000 - 230,000; Full time

Job summary

General Analysis is seeking a Research Engineer to build the systems that define how frontier AI models are evaluated and secured. You will iterate on agent frameworks and build the scaffolding that lets models perform at their best, incorporating state-of-the-art tools for task delegation and complex multi-step tasks.

You will design and build evaluation environments, including challenges that measure a model's ability to evade detection.

Qualifications

• Strong production programming skills and ability to move quickly from prototype to reliable infrastructure.
• Experience building agentic systems, evaluation harnesses, or environment frameworks or eagerness to learn.
• Security experience or interest in building security into AI systems.

Responsibilities

• Build the systems that define how frontier AI models are evaluated and secured.
• Iterate on agent frameworks and scaffolding for multi-step tasks.
• Design and build evaluation environments including challenges to test model evasion capabilities.
• Collaborate with red teamers, pentesters, and contracted security firms to ground experiments.
• Own the pipeline end-to-end from API design to analysis and visualization of thousands of agent trajectories.
• Publish results in blog posts and academic venues and deliver findings to customers.

Skills

Production coding

Agentic systems

Evaluation harnesses

Security experience

Analytical skills

Job description

General Analysis is seeking a Research Engineer to build the systems that define how frontier AI models are evaluated and secured. You will iterate on agent frameworks and build the scaffolding that lets models perform at their best, incorporating state-of-the-art tools for task delegation and complex multi-step tasks.

You will design and build evaluation environments, including challenges that measure a model's ability to evade detection.