Senior Software Security Engineer - AI Benchmark Development

Gramian Consulting

Colombia

Remote

COP 60.000.000 - 90.000.000

Vollzeit

vor 23 Stunden
Sei unter den ersten Bewerbenden

Eine vollständige Bewerbung in einer Minute

Maßgeschneiderter Lebenslauf und Anschreiben – versandbereit.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Gramian Consultancy is seeking an experienced Senior Software Security Engineer to develop training and evaluation tasks for advanced AI coding models. You will transform real-world vulnerabilities into Harbor-format audit-and-patch tasks across multiple languages.

This role combines software security research, vulnerability reproduction, benchmark development, automated testing, and technical review. Fully remote contractor engagement offered.

Qualifikationen

  • 5+ years of professional software engineering experience.
  • Expert-level proficiency in at least one listed programming language.
  • Strong understanding of software security principles and common vulnerability classes.
  • Practical knowledge of CWE, CVE, and OWASP Top 10.
  • Ability to independently reproduce vulnerabilities and develop proof-of-vulnerability tests.
  • Hands-on security testing and analysis tooling experience.

Aufgaben

  • Vulnerability Research: source and investigate real-world vulnerabilities from CVEs and advisories.
  • Security Classification: identify and map to CWE and OWASP Top 10.
  • Vulnerability Reproduction: reproduce issues and develop proof-of-vulnerability tests.
  • Benchmark Development: build Harbor-format tasks across languages and ecosystems.
  • Task Authoring: write realistic instructions without exposing code or patches.
  • Environment Setup: create Docker-based reproducible environments.
  • Verifier Development: design automated verifiers to prevent gaming and overfitting.
  • Patch Evaluation: review AI-generated patches with structured rubrics.
  • Quality Assurance: avoid superficial pattern matching.
  • Technical Documentation: document context, steps, and remediation.

Kenntnisse

Python
JavaScript / TypeScript
Java
C / C++
C#
Go
Rust
Ruby
PHP
Kotlin
Swift
Git
Linux
Bash
Docker

Tools

CodeQL
Semgrep
Sanitizers
Fuzzers

Jobbeschreibung

About Gramian

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

About The Role

We are seeking an experienced Senior Software Security Engineer to develop high-quality training and evaluation tasks for advanced AI coding models. You will transform real-world software vulnerabilities, including CVEs, security advisories, and open-source bug reports, into realistic Harbor-format audit-and-patch tasks across multiple programming languages.

This role combines software security research, vulnerability reproduction, benchmark development, automated testing, and technical review. You will create tasks that assess whether AI systems can identify vulnerabilities, understand their underlying causes, reproduce security issues, and implement reliable fixes rather than simply recognizing familiar patterns or copying existing patches.

The ideal candidate has strong professional software engineering experience, deep practical knowledge of application security, and the ability to independently investigate vulnerabilities and build reproducible testing environments.

Engagement Type: Contractor assignment

Duration: 3 months

Start Date: Expected to start next week

Commitment: 40 hours per week

Work Arrangement: Fully remote

Project Focus: AI coding model training and evaluation, software vulnerability research, and secure patch development.

Key Responsibilities
  • Vulnerability Research: Source and investigate real-world vulnerabilities from CVEs, security advisories, bug reports, and open-source repositories
  • Security Classification: Identify and classify vulnerabilities using the most specific applicable CWE and map them to relevant OWASP Top 10 categories where appropriate
  • Vulnerability Reproduction: Reproduce security issues and develop reliable proof-of-vulnerability tests that demonstrate the underlying weakness
  • Benchmark Development: Build Harbor-format audit-and-patch tasks across multiple programming languages and software ecosystems
  • Task Authoring: Write realistic task instructions that describe the security issue without revealing vulnerable code, exploit details, or the expected remediation
  • Environment Setup: Create reproducible Docker-based environments with pinned dependencies, clear setup instructions, and reliable execution procedures
  • Verifier Development: Design and harden automated verifiers to prevent gaming, test manipulation, hardcoded solutions, and overfitting
  • Patch Evaluation: Review and score AI-generated patches and technical reasoning using structured security and engineering rubrics
  • Quality Assurance: Identify tasks that can be solved through superficial pattern matching or copying known upstream patches without understanding the vulnerability
  • Technical Documentation: Document vulnerability context, reproduction steps, expected behavior, patch requirements, test methodology, and review decisions
Requirements
  • 5+ years of professional software engineering experience
  • Expert-level proficiency in at least one of the following programming languages:
    • Python
    • JavaScript / TypeScript
    • Java
    • C / C++
    • C#
    • Go
    • Rust
    • Ruby
    • PHP
    • Kotlin
    • Swift
  • Strong understanding of software security principles and common vulnerability classes
  • Practical knowledge of CWE, CVE, and OWASP Top 10
  • Ability to independently reproduce vulnerabilities and develop proof-of-vulnerability tests
  • Hands-on experience with security testing and analysis tools, including:
    • Sanitizers
    • Fuzzers
    • Static-analysis tools such as CodeQL or Semgrep
  • Strong proficiency with Git, Linux, Bash, and Docker
  • Ability to create automated tests and reproducible technical evaluation environments
  • Excellent technical writing skills, with the ability to explain complex security issues and remediation requirements clearly
  • Ability to work independently in a fully remote, research-oriented environment
Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Senior Software Security Engineer - AI Benchmarking
Senior Software Security Engineer - AI Benchmarking

Gramian Consulting • Kolumbien

Remote
COP 60.000.000 - 90.000.000
Application Security Engineer | $20/hr Remote
Application Security Engineer | $20/hr Remote

Crossing Hurdles • Kolumbien

Remote
COP 76.938.000 - 104.093.000
Sr. AI Security Engineer
Sr. AI Security Engineer

Backblaze • Kolumbien

Vor Ort
COP 285.938.952 - 393.166.059
Computational Life Sciences & Scientific Python Specialist
Computational Life Sciences & Scientific Python Specialist

Gramian Consulting • Kolumbien

Remote
COP 182.954.000 - 365.909.000
Fully remote
Senior AI Security Engineer - Remote, Vulnerability Trainer
Senior AI Security Engineer - Remote, Vulnerability Trainer

Crossing Hurdles • Kolumbien

Remote
COP 76.938.000 - 104.093.000
Senior Software Engineer – Open Source & SWE-Bench Evaluation
Senior Software Engineer – Open Source & SWE-Bench Evaluation

Anyone AI Inc. • Kolumbien

Remote
COP 66.960.000 - 133.920.000
Remote work
Flexible hours
Senior Application Security Developer
Senior Application Security Developer

Techwarely LLC • Kolumbien

Remote
COP 90.000.000 - 130.000.000
Security Engineer
Security Engineer

Arionkoder • Kolumbien

Remote
COP 295.673.000 - 427.084.000
Monthly USD Service Fee
20 business days of paid Agreed Time O
Caregiver Time Off
+1
Program Manager
Program Manager

Jamefy • Kolumbien

Remote
COP 180.000.000 - 240.000.000
Growth without limits
Competitive compensation
Remote work with flexible hours
+3
Security Senior Software Developer
Security Senior Software Developer

Alcor • Bogotá ciudad

Remote
COP 120.000.000 - 180.000.000