Gramian Consulting Group is hiring a Senior Software Security Engineer - AI Benchmark Development (remote, open to people in Pakistan). Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Key responsibilities
- The ideal candidate has strong professional software engineering experience, deep practical knowledge of application security, and the ability to independently investigate vulnerabilities and build reproducible testing environments
- Engagement Type: Contractor assignment Duration: 3 months Start Date: Expected to start next week Commitment: 40 hours per week Work Arrangement: Fully remote Project Focus: AI coding model training and evaluation, software vulnerability research, and secure patch development.
- Vulnerability Research: Source and investigate real-world vulnerabilities from CVEs, security advisories, bug reports, and open-source repositories
- Security Classification: Identify and classify vulnerabilities using the most specific applicable CWE and map them to relevant OWASP Top 10 categories where appropriate
- Vulnerability Reproduction: Reproduce security issues and develop reliable proof-of-vulnerability tests that demonstrate the underlying weakness
- Benchmark Development: Build Harbor-format audit-and-patch tasks across multiple programming languages and software ecosystems
- Task Authoring: Write realistic task instructions that describe the security issue without revealing vulnerable code, exploit details, or the expected remediation
- Environment Setup: Create reproducible Docker-based environments with pinned dependencies, clear setup instructions, and reliable execution procedures
- Verifier Development: Design and harden automated verifiers to prevent gaming, test manipulation, hardcoded solutions, and overfitting
- Patch Evaluation: Review and score AI-generated patches and technical reasoning using structured security and engineering rubrics
Requirements
- 5+ years of professional software engineering experience
- Expert-level proficiency in at least one of the following programming languages
- JavaScript / TypeScript
- Strong understanding of software security principles and common vulnerability classes
- Practical knowledge of CWE, CVE, and OWASP Top 10
- Ability to independently reproduce vulnerabilities and develop proof-of-vulnerability tests
- Hands-on experience with security testing and analysis tools, including
- Static-analysis tools such as CodeQL or Semgrep
- Strong proficiency with Git, Linux, Bash, and Docker
About Gramian Consulting Group
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
Remote details
- Remote, open to people in Pakistan (remote role)
- Ability to work independently in a fully remote, research-oriented environment
- Employment type: Contract
Salary and benefits
The posting does not give a salary figure.