QA Engineer

Smctechsolutions; Pakistan; On-site; PKR 1,200,000 - 2,400,000; Full time

Job summary

Smctechsolutions is seeking a QA engineer to own testing of a product with AI responsibilities. You will test UI, API, and AI workflows, ensuring quality, safety, and reliability of AI agents and automated processes.

Responsibilities include designing test plans for AI tasks, building regression suites, evaluating prompts, and validating integrations end-to-end. You’ll work with Python-based test scripts and non-deterministic AI systems in a fast-paced environment.

Qualifications

• 3+ years in software QA or quality engineering.
• Strong test design skills: functional, regression, edge-case, exploratory.
• Experience testing AI, LLM, chatbot, or conversational AI systems.

Responsibilities

• Build and maintain UI and API test automation.
• Design and run test plans for AI workflows and agent behaviours.
• Build regression suites and golden datasets to re-check behaviour after changes.
• Evaluate prompt and response quality across models: accuracy, consistency, tone, failure modes.
• Test edge cases, adversarial inputs, hallucination risk, and prompt-injection exposure.
• Verify human-in-the-loop checkpoints trigger and escalate correctly as designed.
• Validate end-to-end integrations — APIs, data retrieval, hand-offs, data handling.
• Monitor live AI agents and flag behavioural drift before it reaches end users.
• Log defects clearly, retest fixes, and own the quality gate before handover.

Skills

QA testing

Test automation

Non-deterministic systems

API testing

Bug reporting

Tools

OneReach.ai

Job description

This is a QA engineering role with AI responsibilities layered on top. You’ll test the full product, UI/UX, functionality, and integrations while also assuring the quality, safety, and reliability of the AI agents and automated workflows the team ships.

Testing AI is not the same as testing software. Outputs are probabilistic, behaviour drifts, and a flow that passed yesterday can fail today on the same input. This role covers both worlds and we’re looking for someone who understands the difference and thrives in it.

What You'll Do

• Build and maintain UI and API test automation
• Design and run test plans for AI workflows and agent behaviours
• Build regression suites and golden datasets to re-check behaviour after any change or model swap
• Evaluate prompt and response quality across models — accuracy, consistency, tone, and failure modes
• Test edge cases, adversarial inputs, hallucination risk, and prompt-injection exposure
• Verify that human-in-the-loop checkpoints trigger and escalate correctly as designed
• Validate integrations end to end — APIs, data retrieval, automated hand-offs, and data handling
• Monitor live AI agents and flag behavioural drift before it reaches end users
• Log defects clearly, retest fixes, and own the quality gate before handover

What We're Looking For

• Required
• 3+ years in software QA, test automation, or quality engineering
• Strong test design skills — functional, regression, edge-case, and exploratory
• Comfortable working with non-deterministic systems where "correct" is a range, not a fixed value
• API and integration testing experience with structured data validation
• Clear, precise defect reporting that an engineer can act on without a follow-up call
• Methodical and sceptical — you won't sign something off just to meet a deadline
• Experience testing LLM, chatbot, or conversational AI systems
• Hands-on experience with OneReach.ai or a comparable orchestration platform
• Familiarity with LLM evaluation frameworks and prompt-evaluation tooling
• Test automation scripting in Python or similar
• Awareness of AI safety standards — EU AI Act, ISO 42001, or NIST AI RMF

Tools & Technologies

Technologies, platforms, and tools you’ll be working with.