Senior QA Automation Engineer - AI Validation & LLM Testing

AI71; Abu Dhabi; On-site; AED 240,000 - 360,000; Full time

Benefits offered by this job

Full Relocation Support

Tax-Free Compensation

Job summary

AI71 is seeking a Senior QA Automation Engineer to define how good AI systems must be. You will bridge Agile development and Systems Engineering, building automated testing that validates LLMs and predictive engines against ground truth datasets.

You will lead TRR and FCA readiness, ensuring data integrity across critical enterprise systems and governance artifacts. The role emphasizes non-deterministic testing, regression suites, performance benchmarks, and secure API validation in a

Qualifications

• 5+ years in QA Automation with data-driven/AI agents
• Experience with ML/AI validation in regulated domains desired
• Ability to define probabilistic pass/fail criteria and confidence levels

Responsibilities

• Lead validation/verification strategies for AI systems
• Build automated testing frameworks to validate AI against Ground Truth
• Ensure TRR and FCA readiness and generate digital evidence
• Validate data consistency between AI agents and critical systems (ERP, BOM)
• Design performance tests for latency and throughput using Locust/K6
• Automate API security validation and RBAC checks

Skills

Python (Pytest)

Selenium/Playwright

Requests

DeepEval

TruLens

Locust

K6

Great Expectations

GitLab CI/CD

QE practices (Shift Left)

Test Pyramid

Mono-repo

Tools

Selenium

Playwright

Requests

DeepEval

TruLens

Locust

K6

Great Expectations

GitLab CI/CD

Job description

AI71 is seeking a Senior QA Automation Engineer to define how good AI systems must be. You will bridge Agile development and Systems Engineering, building automated testing that validates LLMs and predictive engines against ground truth datasets.

You will lead TRR and FCA readiness, ensuring data integrity across critical enterprise systems and governance artifacts. The role emphasizes non-deterministic testing, regression suites, performance benchmarks, and secure API validation in a