AI Assurance Quality Engineer

Cognizant; Dubai; On-site; AED 180,000 - 240,000; Full time

Job summary

Cognizant in Dubai is seeking an experienced AI Assurance Quality Engineer to ensure the quality, reliability, and performance of AI-powered conversational systems, including chatbots, virtual assistants, and RAG/agentic AI solutions.

The role focuses on testing non-deterministic AI outputs, developing test automation in Python, and evaluating LLM-based applications across airline/travel domains such as Amadeus Altéa, Sabre, Navitaire, and GDS/NDC environments.

Qualifications

• 4+ years of software quality engineering experience.
• Strong Python scripting skills.
• Experience testing LLM-based and Conversational AI systems.

Responsibilities

• Design and execute test plans for AI-powered chatbots and assistants.
• Evaluate non-deterministic outputs using rubrics and golden datasets.

Skills

Python scripting

Software Quality Engineering

Test automation

LLM testing

Job description

We are looking for an experienced AI Assurance Quality Engineer to ensure the quality, reliability, and performance of AI-powered conversational systems, including chatbots, virtual assistants, RAG applications, and agentic AI solutions.

Key Requirements

• 4+ years of Software Quality Engineering experience
• Strong Python scripting skills with experience building test automation tools/frameworks

Hands-on Experience Testing

• LLM-based applications
• Conversational AI systems
• Chatbots & Virtual Assistants
• RAG and Agentic AI solutions

Strong Understanding Of

• Prompt Engineering
• Context Management
• Function/Tool Calling
• Retrieval Mechanisms
• AI Failure Modes & Quality Risks

Experience Evaluating Non-deterministic AI Outputs Using

• Golden Datasets
• Rubric-Based Assessments
• LLM-as-a-Judge Frameworks
• Human Review Workflows
• Statistical Quality Metrics

Mandatory Domain Experience

Strong Airline/Travel Industry Expertise In

• Amadeus Altéa, Sabre, or Navitaire
• PSS / Reservation Systems
• GDS & NDC
• IATA Fare Construction
• Fare Rules & Ancillaries
• IROPS
• Payments & Refunds

CogAE106