Senior AI QA & Evaluation Engineer

GCS; Abu Dhabi; Hybrid; AED 250,000 - 450,000; Full time

Job summary

GCS in the United Arab Emirates is hiring a Senior QA Automation Engineer to own quality for AI agentic systems, defining what good looks like when a system reasons, calls tools, and can be wrong in subtle ways. You’ll build end-to-end evaluation, and test non-deterministic AI across live production traffic.

You'll also cover API testing, security validations, and performance testing in a hybrid Abu Dhabi setup, collaborating with expert teams in a contract-based onboarding process.

Qualifications

• 5+ years in QA or quality engineering, with 2+ years testing ML, LLM, or AI agent systems.
• Hands-on with LLM/agent evaluation frameworks and LLM-as-judge.
• Agent observability and tracing experience.
• Expert Python with Pytest and API testing.

Responsibilities

• Test agentic systems end to end: tool-use and trajectory evaluation, grounding and citation verification, honest-failure and refusal calibration, guardrail and adversarial testing.
• Build evaluation for non-deterministic AI: scoring for hallucination, consistency, and accuracy against gold datasets.
• Own prompt and model regression suites that catch quality drift before it ships.
• Run continuous evaluation on live production traffic, with quality monitoring, alerting, and incident response.
• Cover fundamentals: integration and API testing, secure gateway validation (RBAC, PII), and performance/load testing.

Skills

Python

Pytest

Selenium

Playwright

API testing

LLM/AI evaluation

Quality engineering

Tools

DeepEval

TruLens

RAGAS

LangSmith

Langfuse

OpenTelemetry

Job description

GCS in the United Arab Emirates is hiring a Senior QA Automation Engineer to own quality for AI agentic systems, defining what good looks like when a system reasons, calls tools, and can be wrong in subtle ways. You’ll build end-to-end evaluation, and test non-deterministic AI across live production traffic.

You'll also cover API testing, security validations, and performance testing in a hybrid Abu Dhabi setup, collaborating with expert teams in a contract-based onboarding process.