Engineering Manager - Software Security & AI Benchmark Delivery

Gramian Consulting

Pakistan

Remote

PKR 22,865,000 - 45,729,000

Full time

8 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Gramian Consulting is seeking an experienced Engineering Manager - Software Security to lead security-focused training and evaluation tasks for frontier AI coding models. You will oversee multiple engineering pods, owning delivery, quality, team capacity, and client alignment while guiding the team through CWE/CVE/OWASP-based evaluations.

The role focuses on transforming vulnerabilities into rigorous tasks, guiding trainers, and ensuring reproducibility and robustness of evaluations for Google

Qualifications

  • 10+ years of software engineering experience with a strong security background.
  • Proven experience leading technical teams with measurable delivery, quality, and schedule targets.
  • Strong understanding of CWE, CVE and OWASP Top 10 across multiple languages and ecosystems.
  • Experience with Git, Linux, Bash and Docker.

Responsibilities

  • Lead multi-pod engineering delivery, ensuring consistent execution and quality.
  • Plan task batches, assign work to Pod Leads, monitor throughput against targets.
  • Manage risk, tooling issues, and dependencies with clients and internal teams.
  • Communicate progress, forecasts, and risks to clients on a regular cadence.
  • Design and review automated tests, verifiers, and reproducible evaluation environments.

Skills

Software security
Team leadership
Project management
Application security
CI/testing
Dashboards/metrics
Docker
Git

Tools

Docker
Git
CI/CD

Job description

About Gramian

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

We are seeking an experienced Engineering Manager - Software Security to lead the development and delivery of security-focused training and evaluation tasks for frontier AI coding models. You will oversee multiple engineering pods, taking ownership of delivery performance, technical quality, team capacity, and client alignment.

The project focuses on transforming real-world software vulnerabilities into rigorous, reproducible tasks that evaluate whether AI models can identify, understand, reproduce, and securely remediate vulnerabilities across different programming languages and software ecosystems.

As the final quality gate, you will ensure that all delivered tasks meet the client's expectations for technical accuracy, difficulty, reproducibility, and evaluation robustness. This position combines software security expertise, multi-team leadership, operational management, and data-driven decision-making, with direct responsibility for delivering against Google's project expectations.

Engagement Type

Engagement Type: Contractor assignment

Duration: 3 months

Start Date: Expected to start next week

Commitment: 40 hours per week

Work Arrangement: Fully remote

Time Zone Requirement: Minimum 4 hours of overlap with PST

Project Focus: Software security, AI coding model evaluation, engineering team management, and benchmark delivery.

Key Responsibilities
  • Multi-Pod Leadership: Own delivery, quality, and capacity across multiple engineering pods, coordinating Pod Leads and ensuring consistent execution
  • Delivery Planning: Plan task batches, assign work to Pod Leads, and monitor daily throughput against phase-specific targets
  • Performance Management: Ensure batches meet Google's expectations for task volume, technical difficulty, quality standards, and delivery timelines
  • Client Communication: Lead weekly client reporting, communicate progress, provide delivery forecasts, and raise risks or dependencies proactively
  • Risk Management: Identify and elevate tooling issues, rate limits, safety blocks, staffing constraints, and other delivery risks
  • Scope Alignment: Work directly with Google to clarify requirements and align on task scope, timelines, task volume, and difficulty changes
  • Final Quality Assurance: Conduct final reviews of security tasks before delivery, ensuring technical accuracy, reproducibility, and resistance to gaming or superficial solutions
  • Technical Calibration: Train and calibrate Pod Leads on CWE classification, verifier robustness, task difficulty, technical quality, and acceptance criteria
  • Recruitment & Hiring: Conduct interviews and make hiring decisions based on approved role requirements and project capacity needs
  • Talent Development: Identify high-performing Trainers and support their development and promotion into Pod Lead positions
  • Capacity Forecasting: Use throughput, workload, and quality data to forecast headcount requirements across project phases
  • Workforce Planning: Initiate hiring waves approximately three weeks ahead of projected capacity requirements
  • Operational Reporting: Use dashboards, performance indicators, and operational metrics to guide delivery decisions and staffing plans
Requirements
  • 10+ years of overall professional software engineering experience, with a strong background in software security
  • Proven experience leading technical teams against measurable delivery, quality, and schedule targets
  • Strong understanding of CWE, CVE, and OWASP Top 10 across multiple programming languages and software ecosystems
  • Demonstrated experience reproducing software vulnerabilities, reviewing security patches, and evaluating proof-of-vulnerability tests
  • Experience designing or reviewing automated tests, verifiers, CI workflows, or reproducible technical evaluation environments
  • Strong proficiency in at least one major programming language, such as:
    • Python
    • JavaScript / TypeScript
    • Java
    • C / C++
    • C#
    • Go
    • Rust
    • Ruby
    • PHP
    • Kotlin
    • Swift
  • Experience with Git, Linux, Bash, Docker, and common software security tooling
  • Ability to use dashboards, throughput metrics, quality data, and operational reports to make delivery and staffing decisions
  • Experience managing or mentoring technical teams, including assigning work, reviewing output, providing feedback, and developing team leads
  • Excellent written and verbal communication skills, including the ability to explain technical risks and project status to clients and internal stakeholders
  • Strong organizational and prioritization skills, with the ability to manage multiple pods, competing priorities, and delivery phases simultaneously
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Engineering Manager - Security & AI Benchmark
Remote Engineering Manager - Security & AI Benchmark

Gramian Consulting • Pakistan

Remote
PKR 22,865,000 - 45,729,000
Senior Software Security Engineer - AI Benchmark Development
Senior Software Security Engineer - AI Benchmark Development

Gramian Consulting • Pakistan

Remote
PKR 22,656,000 - 35,242,000
Engineering at Gramian Consulting Group, Remote Pakistan NEW
Engineering at Gramian Consulting Group, Remote Pakistan NEW

The Briefed • Pakistan

Remote
PKR 3,200,000 - 7,200,000
Engineering Manager: Software Security & AI Benchmark
Engineering Manager: Software Security & AI Benchmark

The Briefed • Pakistan

Remote
PKR 3,200,000 - 7,200,000
Lead Software at Gramian Consulting Group, Remote Pakistan NEW
Lead Software at Gramian Consulting Group, Remote Pakistan NEW

The Briefed • Pakistan

Remote
PKR 11,065,000 - 22,130,000
Senior at Gramian Consulting Group, Remote Pakistan NEW
Senior at Gramian Consulting Group, Remote Pakistan NEW

The Briefed • Pakistan

Remote
PKR 4,000,000 - 7,000,000
Principal SecOps Consultant - Afternoon Shift
Principal SecOps Consultant - Afternoon Shift

10Pearls • Karachi Division

On-site
PKR 33,195,000 - 49,793,000
Remote Senior Software Security Engineer — AI Benchmarking
Remote Senior Software Security Engineer — AI Benchmarking

Gramian Consulting • Pakistan

Remote
PKR 22,656,000 - 35,242,000
Senior Software Security Engineer AI Benchmark Lead Remote
Senior Software Security Engineer AI Benchmark Lead Remote

The Briefed • Pakistan

Remote
PKR 11,065,000 - 22,130,000
Senior Software Security Engineer - AI Benchmarking Remote
Senior Software Security Engineer - AI Benchmarking Remote

The Briefed • Pakistan

Remote
PKR 4,000,000 - 7,000,000
Top locations