Cybersecurity Engineer - AI Benchmarking

Gramian Consulting Group

Indonesia

Remote

IDR 240,000,000 - 420,000,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Gramian Consultancy seeks a Senior Software Security Engineer to develop AI benchmarking tasks for advanced coding models. You will transform real-world vulnerabilities into Harbor-format audit-and-patch tasks across languages, focusing on vulnerability reproduction and reliable fixes.

Role combines security research, benchmark development, automated testing, and technical reviews. Requires 5+ years in software engineering and deep knowledge of CWE/CVE and OWASP Top 10.

Qualifications

  • 5+ years of professional software engineering experience.
  • Expert-level proficiency in at least one programming language from the listed options.

Responsibilities

  • Source and investigate real-world vulnerabilities from CVEs, advisories, bug reports, and open-source repos.
  • Identify and classify vulnerabilities using CWE and map to OWASP Top 10 where applicable.
  • Reproduce security issues and develop proof-of-vulnerability tests.
  • Build Harbor-format audit-and-patch tasks across multiple languages and ecosystems.
  • Write realistic task instructions without revealing vulnerable code or remediation details.
  • Create reproducible Docker-based environments with pinned dependencies and setup instructions.
  • Design and harden automated verifiers to prevent gaming or manipulation.
  • Review AI-generated patches using structured security rubrics.
  • Identify tasks solvable by superficial pattern matching or copying upstream patches.
  • Document vulnerability context, reproduction steps, patch requirements, and test methodology.

Skills

Python
Java
C/C++
C#
Go
Rust
Ruby
PHP
Kotlin
Swift

Tools

CodeQL
Semgrep
Git
Linux
Bash
Docker

Job description

Cybersecurity Engineer - AI Benchmarking

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

About the Role

We are seeking an experienced Senior Software Security Engineer to develop high-quality training and evaluation tasks for advanced AI coding models. You will transform real-world software vulnerabilities, including CVEs, security advisories, and open-source bug reports, into realistic Harbor-format audit-and-patch tasks across multiple programming languages.

This role combines software security research, vulnerability reproduction, benchmark development, automated testing, and technical review. You will create tasks that assess whether AI systems can identify vulnerabilities, understand their underlying causes, reproduce security issues, and implement reliable fixes rather than simply recognizing familiar patterns or copying existing patches.

The ideal candidate has strong professional software engineering experience, deep practical knowledge of application security, and the ability to independently investigate vulnerabilities and build reproducible testing environments.

Engagement Type: Contractor assignment
Duration: 3 months
Start Date: Expected to start next week
Commitment: 40 hours per week
Work Arrangement: Fully remote
Project Focus: AI coding model training and evaluation, software vulnerability research, and secure patch development.

Key Responsibilities
  • Vulnerability Research: Source and investigate real-world vulnerabilities from CVEs, security advisories, bug reports, and open-source repositories.
  • Security Classification: Identify and classify vulnerabilities using the most specific applicable CWE and map them to relevant OWASP Top 10 categories where appropriate.
  • Vulnerability Reproduction: Reproduce security issues and develop reliable proof-of-vulnerability tests that demonstrate the underlying weakness.
  • Benchmark Development: Build Harbor-format audit-and-patch tasks across multiple programming languages and software ecosystems.
  • Task Authoring: Write realistic task instructions that describe the security issue without revealing vulnerable code, exploit details, or the expected remediation.
  • Environment Setup: Create reproducible Docker-based environments with pinned dependencies, clear setup instructions, and reliable execution procedures.
  • Verifier Development: Design and harden automated verifiers to prevent gaming, test manipulation, hardcoded solutions, and overfitting.
  • Patch Evaluation: Review and score AI-generated patches and technical reasoning using structured security and engineering rubrics.
  • Quality Assurance: Identify tasks that can be solved through superficial pattern matching or copying known upstream patches without understanding the vulnerability.
  • Technical Documentation: Document vulnerability context, reproduction steps, expected behavior, patch requirements, test methodology, and review decisions.
Requirements

5+ years of professional software engineering experience.

Expert-level proficiency in at least one of the following programming languages:

Python

Java

C / C++

C#

Go

Rust

Ruby

PHP

Kotlin

Swift

Strong understanding of software security principles and common vulnerability classes.

Practical knowledge of CWE, CVE, and OWASP Top 10.

Ability to independently reproduce vulnerabilities and develop proof-of-vulnerability tests.

Hands-on experience with security testing and analysis tools, including:

Static-analysis tools such as CodeQL or Semgrep

Strong proficiency with Git, Linux, Bash, and Docker.

Ability to create automated tests and reproducible technical evaluation environments.

Excellent technical writing skills, with the ability to explain complex security issues and remediation requirements clearly.

Ability to work independently in a fully remote, research-oriented environment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Cybersecurity Engineer - AI Benchmarking Lead
Cybersecurity Engineer - AI Benchmarking Lead

Gramian Consulting Group • Indonesia

Remote
IDR 1,510,526,000 - 3,021,053,000
Senior Cybersecurity Engineer - AI Benchmarking & Vulnerability
Senior Cybersecurity Engineer - AI Benchmarking & Vulnerability

Gramian Consulting Group • Indonesia

Remote
IDR 240,000,000 - 420,000,000
Lead Cybersecurity Engineer - Remote AI Benchmarking
Lead Cybersecurity Engineer - Remote AI Benchmarking

Gramian Consulting Group • Indonesia

Remote
IDR 1,510,526,000 - 3,021,053,000
Application Security Engineer | $20/hr Remote
Application Security Engineer | $20/hr Remote

Crossing Hurdles • Indonesia

Remote
IDR 414,033,000 - 589,683,000
Senior Application Security Engineer - GenAI Security Evaluation
Senior Application Security Engineer - GenAI Security Evaluation

Gramian Consulting Group • Indonesia

Remote
IDR 2,169,590,000 - 3,254,384,000
Cyber Security Specialist
Cyber Security Specialist

Codefeast • Indonesia

On-site
IDR 500,000,000 - 800,000,000
Engineering Manager – Software Security
Engineering Manager – Software Security

Codefeast • Indonesia

On-site
IDR 420,000,000 - 640,000,000
Remote Senior Software Security Engineer - AI Trainer
Remote Senior Software Security Engineer - AI Trainer

Crossing Hurdles • Indonesia

Remote
IDR 414,033,000 - 589,683,000
AI Security Evaluator & Attack-Scenario Architect
AI Security Evaluator & Attack-Scenario Architect

Codefeast • Indonesia

On-site
IDR 500,000,000 - 800,000,000
Senior Security Operation
Senior Security Operation

Glints • Batam

On-site
IDR 300,000,000 - 700,000,000