Research Engineer, Evaluations

General Analysis; San Francisco, Northern (CA, KY); Hybrid; USD 150,000 - 230,000; Full time

Job summary

General Analysis is seeking a Research Engineer to build the systems that define how frontier AI models are evaluated and secured. You will iterate on agent frameworks and build the scaffolding that lets models perform at their best, incorporating state-of-the-art tools for task delegation and complex multi-step tasks.

You will design and build evaluation environments, including challenges that measure a model's ability to evade detection.

Qualifications

• Strong production programming skills and ability to move quickly from prototype to reliable infrastructure.
• Experience building agentic systems, evaluation harnesses, or environment frameworks or eagerness to learn.
• Security experience or interest in building security into AI systems.

Responsibilities

• Build the systems that define how frontier AI models are evaluated and secured.
• Iterate on agent frameworks and scaffolding for multi-step tasks.
• Design and build evaluation environments including challenges to test model evasion capabilities.
• Collaborate with red teamers, pentesters, and contracted security firms to ground experiments.
• Own the pipeline end-to-end from API design to analysis and visualization of thousands of agent trajectories.
• Publish results in blog posts and academic venues and deliver findings to customers.

Skills

Production coding

Agentic systems

Evaluation harnesses

Security experience

Analytical skills

Job description

General Analysis is a frontier security lab. We protect the world as AI systems grow more capable and more dangerous in the wrong hands. We build high-fidelity research infrastructure that evaluates and strengthens frontier models on critical cyberdefense tasks, and we bring that research directly to enterprises — securing the AI systems they deploy through adversarial testing and runtime protection.

About the role

As a Research Engineer, you will build the systems that define how frontier AI models are evaluated and secured.

You will iterate on our agent frameworks and build the scaffolding that lets models perform at their best, incorporating state-of-the-art tools for task delegation and complex multi-step tasks.

You will design and build evaluation environments, including challenges that measure a model's ability to evade detection.

You will work with expert red teamers, pentesters, and contracted security firms to ground these environments in real-world adversarial practice.

You will own the pipeline end to end, from designing APIs for frontier labs to building the analysis and visualization tools that summarize 10,000+ agent trajectories into specific conclusions.

From time to time, you will publish results in blog posts and academic venues, and deliver findings directly to customers.

You may be a good fit if

• You have strong production programming skills and can move quickly from prototype to reliable infrastructure.
• You have experience building agentic systems, evaluation harnesses, or environment frameworks, or are eager to learn.
• You have security experience, or are excited to build it here.
• You have strong problem-solving and analytical skills, and can extract clear signal from messy data.

You'll thrive here if

• You are excited about applying novel research and new technology to real-world problems.
• You are excited to learn new domains and build deep context in AI security.
• You are results-oriented and can balance deep exploration with practical implementation.
• You proactively step outside defined boundaries to support the team's mission and unblock others.