AI Quality Engineer

Rakbank; United Arab Emirates; On-site; AED 360,000 - 600,000; Full time

Job summary

Rakbank in the United Arab Emirates seeks an experienced Assistant Manager – Automation and AI Engineer to lead quality engineering across Generative, Conversational, and Intelligent Automation initiatives. This hands-on role ensures AI-powered solutions are accurate, secure, and production-ready.

Responsibilities include defining AI testing strategies, benchmarking LLMs, building automated tests with Python, PyTest and Playwright, and mentoring QA teams while partnering with Product,

Qualifications

• Bachelor's degree in a relevant field; 8–10 years testing experience with banking tech exposure preferred.

Responsibilities

• Lead end-to-end quality engineering for Generative AI, Conversational AI, Agentic AI, and intelligent automation solutions.

Skills

Automation testing for AI systems

Python

PyTest

Playwright

API automation

CI/CD

Mentoring

Stakeholder management

English/Arabic bilingual

Education

Bachelor's Degree

Tools

UiPath Test Suite

Azure DevOps

Jenkins

GitLab

Git

Job description

Job Description

Rakbank is seeking an experienced Assistant Manager – Automation and AI Engineer to lead quality engineering and testing across our AI, Generative AI, Conversational AI, Agentic AI, and Intelligent Automation initiatives. This hands‑on role will play a critical part in ensuring AI‑powered solutions are accurate, reliable, secure, explainable, and production‑ready while driving automation excellence across the Digital & AI ecosystem.

What You Will Do

• Lead end-to-end quality engineering for Generative AI, Conversational AI, Agentic AI, and intelligent automation solutions.
• Define AI testing strategies, evaluation frameworks, quality standards, and release‑readiness criteria.
• Evaluate and benchmark Large Language Models (LLMs) for accuracy, safety, relevance, latency, groundedness, and cost.
• Design and implement AI testing frameworks covering RAG evaluation, hallucination detection, AI safety, prompt validation, and model regression.
• Build and integrate automated testing solutions using Python, PyTest, Playwright, API automation, and CI/CD quality gates.
• Drive chatbot, virtual assistant, and copilot testing across multi‑turn conversations, intent coverage, and English/Arabic user journeys.
• Mentor QA and automation teams while partnering with Product, Engineering, and Data Science teams to embed quality throughout the AI development lifecycle.

What You Will Bring

• Bachelor's Degree plus 8-10 years of testing experience, including at least 3 years supporting banking technology projects in a lead or SME capacity.
• Strong expertise in Generative AI, Conversational AI, Agentic AI testing, LLM evaluation, RAG frameworks, and AI quality assurance methodologies.
• Hands‑on experience with Python, PyTest, Playwright, API automation, UiPath Test Suite, and intelligent automation testing.
• Knowledge of AI safety testing, prompt injection, jailbreak testing, hallucination detection, privacy controls, and guardrail validation.
• Experience integrating automated testing and AI evaluation into Azure DevOps, Jenkins, GitLab, and modern CI/CD pipelines.
• Strong analytical, stakeholder management, problem‑solving, and mentoring capabilities, with knowledge of banking regulations, UAE PDPL, and digital banking environments.

Preferred Skills

• Experience in Generative AI, Conversational AI, Agentic AI, and intelligent automation testing.
• Strong expertise in LLM evaluation, benchmarking, prompt validation, model regression, and RAG evaluation frameworks.
• Hands‑on experience in AI quality assurance, including hallucination detection, LLM-as‑a‑Judge frameworks, and AI safety testing.
• Proficiency in Python, PyTest, Playwright, API automation, and UiPath Test Suite for automation and RPA testing.
• Experience integrating automated testing into CI/CD pipelines using Azure DevOps, Jenkins, GitLab, and Git.
• Knowledge of test data management, synthetic data generation, performance testing, latency profiling, and AI observability.
• Strong understanding of Azure DevOps Test Plans, Jira, and modern AI concepts such as embeddings, vector databases, context windows, and orchestration frameworks.
• Banking domain knowledge with awareness of UAE PDPL, data privacy, security, and regulatory compliance requirements.