Cybersecurity Environments Engineer for AI Evaluation

Collinear AI; Sunnyvale (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Collinear AI in Sunnyvale is building cybersecurity environments, tasks, and evaluations to help frontier AI models improve at real work. We are growing our security team and seeking hands-on security experts who can turn practical problems into runnable environments, with verifiable fixes and scalable testing.

You will own work from initial scenario through runnable environment, task instructions, reference solution, and verifier.

Qualifications

• Hands-on security expertise across cybersecurity domains.
• Ability to read unfamiliar code, reproduce issues, and implement fixes.
• Comfort with Linux, Git, containers, debugging, and automated testing.
• Clear technical writing and careful judgment on evaluation realism and trustworthiness.
• Curiosity about applying cybersecurity to AI; prior LLM experience not required.

Responsibilities

• Build cybersecurity environments to host realistic tasks and assessments.
• Turn security work into clearly defined agent tasks and objectives.
• Develop reference solutions and verifiers to validate fixes.
• Test the tests and guard against shortcuts and hidden solutions.
• Run and analyze agents, reviewing tool calls and changes for gaps.
• Contribute to benchmarks and research with researchers and engineers.

Skills

Practical cybersecurity

Linux

Git

Containers

Debugging

Automated testing

Technical writing

Curiosity about AI/LLMs

Job description

Collinear AI in Sunnyvale is building cybersecurity environments, tasks, and evaluations to help frontier AI models improve at real work. We are growing our security team and seeking hands-on security experts who can turn practical problems into runnable environments, with verifiable fixes and scalable testing.

You will own work from initial scenario through runnable environment, task instructions, reference solution, and verifier.