AI Quality & Evaluation Engineer for Secure AI Systems

Delen Private Bank; Antwerpen; On-site; EUR 60,000 - 80,000; Full time

Benefits offered by this job

Hospitalisation insurance

Dental insurance

Group insurance

Mobility budget

Job summary

Delen Private Bank is seeking a QA & AI Evaluation Engineer to safeguard the quality, reliability, and safety of the AIHub Platform. You will define evaluation criteria, build datasets, and automate evaluations integrated into delivery pipelines.

Collaboration with AI engineers, developers, analysts and stakeholders is essential to translate business expectations into measurable criteria. You will design evaluation strategies, run tests on APIs, data processing and UIs, and monitor model

Qualifications

• Experience as a QA Engineer in an agile IT environment
• Hands-on testing or evaluation of LLM applications
• Ability to translate business expectations into measurable evaluation criteria
• Familiarity with non-deterministic AI systems and edge cases
• Strong analytical and structured thinking
• Excellent communication across technical and non-technical stakeholders

Responsibilities

• Translate AI behaviour expectations into measurable acceptance criteria
• Define evaluation datasets including edge cases and failure scenarios
• Develop deterministic checks: format validation, schema checks, citations, correct tool calls
• Design and calibrate LLM-as-a-judge evaluations against human judgments
• Run functional and non-functional tests on platform components
• Monitor quality in production and track regression after model upgrades
• Write annotation guidelines and review labelling outputs
• Communicate findings clearly to stakeholders

Skills

QA engineering

AI evaluation

Analytical thinking

Team collaboration

Tools

Python

LLM evaluation frameworks

Postman/OpenAPI

Azure DevOps

SQL/JSON

Job description

Delen Private Bank is seeking a QA & AI Evaluation Engineer to safeguard the quality, reliability, and safety of the AIHub Platform. You will define evaluation criteria, build datasets, and automate evaluations integrated into delivery pipelines.

Collaboration with AI engineers, developers, analysts and stakeholders is essential to translate business expectations into measurable criteria. You will design evaluation strategies, run tests on APIs, data processing and UIs, and monitor model