Remote Security Software Engineer (Python) for AI Evaluation

Braintrust; United States; Remote; USD 107,453,000 - 128,943,000; Part time

Job summary

Leading AI Lab is seeking an experienced software engineer for security-focused coding task evaluation and prompt creation. This contracting engagement involves evaluating vulnerabilities, validating exploits, and writing clear feedback, with potential for longer-term work.

Remote candidates in the United States, Canada and 21+ other countries are eligible; expected 20 hours per week at $75–$90/hr, requiring 5+ years of software development and strong Python skills.

Qualifications

• At least five years of professional software-development experience and strong Python skills.
• Hands-on experience with vulnerability research, exploit reproduction or verification, or implementing, backporting or validating security patches.
• The ability to apply structured evaluation criteria and write clear technical feedback.
• Fluency in written and spoken English.

Responsibilities

• Evaluate coding tasks involving software vulnerabilities, exploit verification and security patches.
• Create high-quality coding prompts and reference answers for benchmark-style problems.
• Evaluate model outputs for code generation, refactoring, debugging and implementation.
• Identify and document model failures, edge cases and reasoning gaps.
• Compare private language models with leading external models.
• Build or configure coding environments for evaluation and reinforcement learning.
• Follow detailed annotation and evaluation guidelines consistently.

Skills

Python

Vulnerability research

Security patching

Technical feedback

English communication

Job description

Leading AI Lab is seeking an experienced software engineer for security-focused coding task evaluation and prompt creation. This contracting engagement involves evaluating vulnerabilities, validating exploits, and writing clear feedback, with potential for longer-term work.

Remote candidates in the United States, Canada and 21+ other countries are eligible; expected 20 hours per week at $75–$90/hr, requiring 5+ years of software development and strong Python skills.