Senior QA Automation Engineer - AI Validation Lead

ai71 Careers Site; Abu Dhabi; On-site; AED 250,000 - 420,000; Full time

Job summary

AI71 is seeking a Senior QA Automation Engineer in Abu Dhabi to lead validation and verification for the company''s AI transformation within EDGE Group collaborations. You will define what good looks like for non-deterministic AI systems and ensure that LLMs and predictive engines meet strict defense standards.

You will bridge Agile development and formal Systems Engineering, building automated testing frameworks that verify software functionality and validate AI behaviors against Ground Truth

Qualifications

• 5+ years of QA automation experience in data-driven or AI-enabled apps.
• Experience in defense, aerospace, or regulated industries is a strong plus.
• Ability to define pass/fail criteria for probabilistic systems and communicate confidence levels.

Responsibilities

• Lead AI/LLM validation and develop automated testing frameworks.
• Assess non-deterministic outputs for hallucination, consistency, and accuracy against ground truth datasets.
• Implement automated metrics to verify RAG pipelines and citation quality.
• Design regression suites to monitor prompt drift and ensure model integrity.
• Build integration tests for ERP and data systems (e.g., SAP S/4HANA, Ariba).
• Create performance tests for forecasting models and risk scoring engines.
• Automate API gateway tests with RBAC and PII redaction checks.
• Align tests with system requirements and user needs for V&V reports.
• Prepare Stage Gate test readiness packages with quantitative evidence.
• Manage defect lifecycle linking to model versions and data sets.

Education

Bachelor's degree in CS/engineering

Tools

Python

Selenium

Playwright

Locust

JMeter

SQL

Great Expectations

Git/GitLab

Jira

CI/CD

Job description

AI71 is seeking a Senior QA Automation Engineer in Abu Dhabi to lead validation and verification for the company''s AI transformation within EDGE Group collaborations. You will define what good looks like for non-deterministic AI systems and ensure that LLMs and predictive engines meet strict defense standards.

You will bridge Agile development and formal Systems Engineering, building automated testing frameworks that verify software functionality and validate AI behaviors against Ground Truth