Senior AI Chatbot QA Automation Engineer

DXC Technology; Dubai; On-site; AED 180,000 - 240,000; Full time

Job summary

DXC Technology in Dubai is seeking an experienced Automation Test Engineer to validate AI-powered customer support and troubleshooting solutions, focusing on test automation, API testing, and AI quality validation.

You will measure AI performance, build automated tests for chatbots, integrate tests into Azure DevOps pipelines, and collaborate with AI engineers and product teams to ensure reliable, scalable testing across platforms.

Qualifications

• 4+ years of QA Automation or SDET experience.
• 1+ year of experience testing AI, Chatbots, LLMs, or Agentic AI solutions.
• Strong programming skills in Python, TypeScript, or C#.
• Experience with REST APIs, GraphQL, WebSockets, Postman, and WireMock.
• Experience with Azure DevOps and CI/CD pipelines.
• Knowledge of RAG, semantic search, and vector databases.
• Experience with MQTT or IoT telemetry is a plus.
• Exposure to embedded systems or consumer electronics is preferred.

Responsibilities

• Develop automated tests for AI chatbots and RAG systems.
• Validate response accuracy, relevance, and AI performance.
• Measure AI quality metrics such as hallucination rates and answer relevance.
• Perform regression testing when AI models or knowledge bases updated.
• Automate testing of troubleshooting workflows and decision trees.
• Simulate customer interactions and verify expected outcomes.
• Build and execute automated API tests.
• Validate integrations between AI platforms and backend systems.
• Create mock services and test data where required.
• Conduct load and performance testing of conversational AI systems.
• Test AI guardrails, prompt injection risks, and unsafe responses.
• Identify performance and reliability issues.
• Integrate automated tests into Azure DevOps pipelines.
• Track quality metrics and generate test reports.Work closely with developers, AI engineers, and product teams.

Skills

Python

TypeScript

C#

REST APIs

GraphQL

WebSockets

Postman

WireMock

Azure DevOps

CI/CD pipelines

RAG

Semantic search

Vector databases

MQTT

Embedded systems

Tools

Postman

WireMock

Azure DevOps

Job description

DXC Technology in Dubai is seeking an experienced Automation Test Engineer to validate AI-powered customer support and troubleshooting solutions, focusing on test automation, API testing, and AI quality validation.

You will measure AI performance, build automated tests for chatbots, integrate tests into Azure DevOps pipelines, and collaborate with AI engineers and product teams to ensure reliable, scalable testing across platforms.