Agentic AI Quality Engineer — MLOps & Safety

Accenture the Netherlands; Amsterdam; Hybrid; EUR 90,000 - 120,000; Full time

Benefits offered by this job

4x9 Workweek Option

Hybrid Working

FlexDay

Competitive Base Salary

MyBenefits Budget

Annual Bonus

Net Allowance

Employee Share Purchase Plan

Collective Health Insurance Scheme

Mental Health Support

Well-being Hub

26 Vacation Days

Special Leave Types

Culture Days

NS Business Card

Mobility Budget

Performance Achievement Program

Learning & Development Platform

Guidance from People Leads

Do Good Volunteer Program

I&D Networks

Refugee Talent Track

Accentrics & Friday Drinks

Annual Accenture NL Party

Flat Organizational Hierarchy

Job summary

Accenture the Netherlands is seeking a Senior ML QA Engineer to lead quality and performance evaluation across conversational AI and multi-step workflows. You will work with quality engineers, software engineers, AI specialists, and business consultants to embed testing in CI/CD, monitor telemetry, and drive safe, compliant decisions for client projects.

You will contribute to governance, observability, and evaluation strategy while mentoring teams and sharing reusable assets across engagements.

Qualifications

• 3+ years in machine learning QA, AI evaluation engineering, or MLOps engineering.
• Bachelor’s or Master’s degree in Computer Science, Data Science, or a related quantitative field.
• Dutch language proficiency (B2+) for collaboration with Dutch-speaking clients and stakeholders; English at C2 level.
• Strong knowledge of prompt engineering, evaluation tools, and telemetry.

Responsibilities

• Automate evaluation of multi-turn conversations and long-horizon tasks using metrics for reasoning accuracy, evidence grounding, plan execution, latency, token usage, and recovery.
• Apply schema validation, tracing, and calibrated judges to detect inaccurate tool calls, loops, and misuse.
• Run red-team, prompt-injection, jailbreak, privacy, and PII-leak tests across agent workflows.
• Embed test harnesses in CI/CD and monitor production telemetry for regressions, behavioral drift, and recovery.
• Translate business risks into measurable quality criteria, practical controls, and release recommendations.
• Communicate findings and residual risks for go/no-go decisions; coach teams and share reusable assets and standards.

Skills

Agentic AI

Python

Prompt engineering

Vector databases

Execution telemetry

Security mindset

Leadership

Education

Bachelor’s or Master’s in Computer Science, Data Science, or related quantitative field

Tools

MLflow

Galileo

PyRIT

Job description

Accenture the Netherlands is seeking a Senior ML QA Engineer to lead quality and performance evaluation across conversational AI and multi-step workflows. You will work with quality engineers, software engineers, AI specialists, and business consultants to embed testing in CI/CD, monitor telemetry, and drive safe, compliant decisions for client projects.

You will contribute to governance, observability, and evaluation strategy while mentoring teams and sharing reusable assets across engagements.