AI Engineer

XPT Software; City of Melbourne; On-site; AUD 150,000 - 210,000; Full time

Job summary

XPT Software in Melbourne is seeking a Senior AI/DevSecOps engineer to design and build an AI/agentic layer for SAST/SCA workflows. You will integrate GitLab findings, develop LLM-powered triage and remediation guidance, and implement guardrails with human-in-the-loop approvals.

You will collaborate with security SMEs and the BA to translate outcomes like reduced MTTR and lower false positives into automated targets, while ensuring data governance and deployment options meet residency

Qualifications

• Hands-on experience with LLM APIs (OpenAI, Anthropic, or equivalent) - function calling/tool use, structured outputs, and context management at production scale.
• Practical experience designing agentic systems - multi-step reasoning, tool-use orchestration, and building in human-approval checkpoints rather than fully autonomous action.
• Strong software engineering skills - able to build and maintain integration pipelines (GitLab API/webhooks, CI/CD hooks) rather than just prototyping in a notebook.
• Understanding of prompt engineering best practices and evaluation methodology (how to measure whether an AI triage decision is actually good, not just plausible-sounding).
• Working knowledge of AppSec concepts (SAST/SCA findings, CVE/CWE, vulnerability severity) sufficient to build meaningful automation.
• Awareness of data governance/privacy implications of sending code and vulnerability data to third-party LLM APIs, and experience implementing controls (redaction, on-prem/private deployment options, data retention policies).
• Familiarity with GitLab's API and webhook model for building external integrations.

Responsibilities

• Design and build integrations that pull SAST/SCA findings out of GitLab (via API/webhooks) into an external pipeline for AI-assisted processing.
• Build LLM-based triage assistance: false-positive likelihood scoring, vulnerability explanation in plain language, and contextual remediation suggestions generated against the actual code diff/repo context.
• Design agentic workflows (multi-step, tool-using LLM agents) that can, for example, correlate a dependency CVE with actual usage in code, check for available safe upgrade paths, and draft a remediation MR for human review.
• Own prompt engineering, evaluation, and guardrails for these agents - including hallucination checks, human-in-the-loop approval gates before any auto-generated fix reaches a merge request, and audit logging for every AI-assisted decision.
• Select and integrate the LLM provider/stack (OpenAI API, or alternatives) factoring in data residency and confidentiality constraints - telco source code and vulnerability data leaving the environment via a third-party API is a governance concern requiring explicit sign-off.
• Build feedback loops so agent outputs (false-positive calls, fix suggestions) are measured against actual analyst/developer decisions to improve accuracy over time.
• Work with the SME to ensure the AI layer complements rather than duplicates native GitLab scanning logic, and with the BA to translate desired outcomes (reduced MTTR, lower false-positive rate) into technical automation targets.
• Document architecture, data flows, and model usage for security/compliance review - critical in a regulated telco environment where any AI system touching source code or vulnerability data will likely need a formal risk assessment.

Skills

LLM API usage

Agentic system design

Integration pipelines

Prompt engineering

AppSec knowledge

Data governance & privacy

GitLab API/webhooks

Education

Bachelor's degree in Computer Science or related field

Tools

GitLab API/Webhooks

OpenAI API

Anthropic API

Self-hosted LLMs

Job description

Build the AI/agentic layer on top of the SAST/SCA capability . GitLab's own Duo/agentic features are out of scope for this programme, so this role is responsible for designing and building custom automation using OpenAI (or equivalent LLM APIs) and agentic patterns to reduce triage effort, cut false-positive noise, and speed up remediation

Role Purpose

Build the AI/agentic layer on top of the SAST/SCA capability . GitLab's own Duo/agentic features are out of scope for this programme, so this role is responsible for designing and building custom automation using OpenAI (or equivalent LLM APIs) and agentic patterns to reduce triage effort, cut false-positive noise, and speed up remediation

Key Responsibilities

• Design and build integrations that pull SAST/SCA findings out of GitLab (via API/webhooks) into an external pipeline for AI-assisted processing.
• Build LLM-based triage assistance: false-positive likelihood scoring, vulnerability explanation in plain language, and contextual remediation suggestions generated against the actual code diff/repo context.
• Design agentic workflows (multi-step, tool-using LLM agents) that can, for example, correlate a dependency CVE with actual usage in code, check for available safe upgrade paths, and draft a remediation MR for human review.
• Own prompt engineering, evaluation, and guardrails for these agents - including hallucination checks, human-in-the-loop approval gates before any auto-generated fix reaches a merge request, and audit logging for every AI-assisted decision.
• Select and integrate the LLM provider/stack (OpenAI API, or alternatives) factoring in data residency and confidentiality constraints - telco source code and vulnerability data leaving the environment via a third-party API is a governance concern requiring explicit sign-off.
• Build feedback loops so agent outputs (false-positive calls, fix suggestions) are measured against actual analyst/developer decisions to improve accuracy over time.
• Work with the SME to ensure the AI layer complements rather than duplicates native GitLab scanning logic, and with the BA to translate desired outcomes (reduced MTTR, lower false-positive rate) into technical automation targets.
• Document architecture, data flows, and model usage for security/compliance review - critical in a regulated telco environment where any AI system touching source code or vulnerability data will likely need a formal risk assessment.

Experience Level

Senior, 8+ years software/AI engineering experience, with at least 1-2 years hands-on building production LLM-based or agentic systems. AppSec domain experience is a strong plus.

Required Knowledge & Skills

• Hands-on experience with LLM APIs (OpenAI, Anthropic, or equivalent) - function calling/tool use, structured outputs, and context management at production scale.
• Practical experience designing agentic systems - multi-step reasoning, tool-use orchestration, and building in human-approval checkpoints rather than fully autonomous action.
• Strong software engineering skills - able to build and maintain integration pipelines (GitLab API/webhooks, CI/CD hooks) rather than just prototyping in a notebook.
• Understanding of prompt engineering best practices and evaluation methodology (how to measure whether an AI triage decision is actually good, not just plausible-sounding).
• Working knowledge of AppSec concepts (SAST/SCA findings, CVE/CWE, vulnerability severity) sufficient to build meaningful automation.
• Awareness of data governance/privacy implications of sending code and vulnerability data to third-party LLM APIs, and experience implementing controls (redaction, on-prem/private deployment options, data retention policies).
• Familiarity with GitLab's API and webhook model for building external integrations.

Nice to Have

• Experience with self-hosted/open-weight LLMs as an alternative to external APIs, for data residency-sensitive use cases.
• Prior experience building AI tooling specifically in a security or DevSecOps context (vulnerability triage, code review automation).
• Exposure to evaluation frameworks/observability tooling for LLM systems in production (tracing, guardrail testing, drift monitoring).