Lead AI QA Engineer - GenAI & Agentic Testing (Airline)

Atain; Abu Dhabi Emirate; On-site; AED 360,000 - 600,000; Full time

Job summary

Atain is seeking an experienced AI Engineer with strong Generative AI, LLMs, AI agents, and AI testing expertise to ensure the quality and safety of AI-powered solutions in the airline industry.

The role focuses on designing robust testing frameworks for LLMs, AI agents, and conversational AI, with emphasis on reliability, security, and performance for airline operations. Immediate joiners or relocations are welcome.

Qualifications

• 8–9+ years of hands-on AI engineering, testing, and QA experience.

Responsibilities

• Develop and support AI applications using Python, Generative AI, and LLMs.
• Design, execute, and automate comprehensive AI test strategies across the AI application lifecycle.
• Validate AI models, LLMs, AI agents, and agentic workflows for accuracy, consistency, reliability, and safety.
• Perform functional, integration, regression, performance, security, and user acceptance testing for AI solutions.
• Conduct LLM evaluation, prompt testing, hallucination testing, bias and fairness assessments, and guardrail validation.
• Develop automated frameworks for testing AI agents, chatbots, copilots, and autonomous workflows.
• Define and track AI quality metrics, benchmarks, and acceptance criteria.
• Identify, analyze, and resolve defects related to AI outputs, workflows, and model behavior.
• Collaborate with business stakeholders and development teams to ensure production-ready AI solutions for airline operations.

Skills

Python

Generative AI

LLMs

AI Testing

AI Agents

Prompt Testing

Hallucination Testing

Agentic AI

Airline Domain

QA Automation

Responsible AI

Education

Bachelor's or Master's in CS/AI/Engineering

Tools

LangChain

LangGraph

CrewAI

AutoGen

LlamaIndex

Semantic Kernel

Job description

Atain is seeking an experienced AI Engineer with strong Generative AI, LLMs, AI agents, and AI testing expertise to ensure the quality and safety of AI-powered solutions in the airline industry.

The role focuses on designing robust testing frameworks for LLMs, AI agents, and conversational AI, with emphasis on reliability, security, and performance for airline operations. Immediate joiners or relocations are welcome.