Lead Agentic AI QA & Safety Engineer

3900 Accenture B.V. Company; Amsterdam; Hybrid; EUR 90,000 - 140,000; Full time

Benefits offered by this job

4x9 Workweek

Hybrid Working

Annual Bonus

Health Insurance

Learning & Development Platform

Mobility Budget

Vacation Days 26

Do Good Volunteer Program

Job summary

Accenture NL seeks an Agentic AI Quality Engineer to lead evaluation and safety testing for next-generation conversational AI. You will join multidisciplinary teams of quality engineers, software engineers, AI specialists, and business consultants to advance reliable, compliant AI solutions.

The role emphasizes defining measurable quality criteria, building reusable assets, and guiding go/no-go decisions with stakeholders across Digital Core and related service lines.

Qualifications

• Education: Bachelor's or Master's in Computer Science, Data Science, or related field.
• Experience: 3+ years in machine learning QA, AI evaluation engineering, or MLOps engineering.
• Languages: Dutch – B2+; English – C2.
• Skills: Agentic AI expertise: evaluating autonomous agents, multi-step workflows, or RAG applications.
• Tech: Strong Python; experience with ML evaluation tools such as MLflow, Galileo, PyRIT.
• Frameworks: Knowledge of prompt engineering, vector databases, tool schema design, and telemetry.
• Security: Hands-on adversarial testing, threat modeling, or model safety guardrails.
• Leadership: Strong communication and growth mindset for innovation.

Responsibilities

• Automate evaluation of multi-turn conversations and long-horizon tasks.
• Implement safety guardrails and human-in-the-loop reviews for high-risk cases.
• Run red-team, prompt-injection, jailbreak, and privacy tests across workflows.
• Embed test harnesses in CI/CD and monitor production telemetry for drift.
• Communicate findings and recommendations to stakeholders and coaches teams.

Skills

Agentic AI

Python

AI evaluation

Adversarial testing

Leadership

Education

Bachelor's/Master's in CS/DS

Tools

MLflow

Galileo

PyRIT

Job description

Accenture NL seeks an Agentic AI Quality Engineer to lead evaluation and safety testing for next-generation conversational AI. You will join multidisciplinary teams of quality engineers, software engineers, AI specialists, and business consultants to advance reliable, compliant AI solutions.

The role emphasizes defining measurable quality criteria, building reusable assets, and guiding go/no-go decisions with stakeholders across Digital Core and related service lines.