AI QA Engineer — Test AI Workflows & Integrations

Smctechsolutions

Pakistan

On-site

PKR 1,200,000 - 2,400,000

Full time

13 hours ago

Be an early applicant

Application generator

Stand out for this role — generate a tailored resume and cover letter in about a minute.

Get past ATS filters

Job summary

Smctechsolutions is seeking a QA engineer to own testing of a product with AI responsibilities. You will test UI, API, and AI workflows, ensuring quality, safety, and reliability of AI agents and automated processes.

Responsibilities include designing test plans for AI tasks, building regression suites, evaluating prompts, and validating integrations end-to-end. You’ll work with Python-based test scripts and non-deterministic AI systems in a fast-paced environment.

Qualifications

3+ years in software QA or quality engineering.
Strong test design skills: functional, regression, edge-case, exploratory.
Experience testing AI, LLM, chatbot, or conversational AI systems.

Responsibilities

Build and maintain UI and API test automation.
Design and run test plans for AI workflows and agent behaviours.
Build regression suites and golden datasets to re-check behaviour after changes.
Evaluate prompt and response quality across models: accuracy, consistency, tone, failure modes.
Test edge cases, adversarial inputs, hallucination risk, and prompt-injection exposure.
Verify human-in-the-loop checkpoints trigger and escalate correctly as designed.
Validate end-to-end integrations — APIs, data retrieval, hand-offs, data handling.
Monitor live AI agents and flag behavioural drift before it reaches end users.
Log defects clearly, retest fixes, and own the quality gate before handover.