Mid-Level QA Engineer (AI & Automation)

Cyber Nest; Lahore; Sur place; PKR 1 200 000 - 1 800 000; Plein temps

Avantages offerts par ce poste

Competitive salary

Growth opportunities

Collaborative team

Résumé du poste

Cyber Nest in Lahore is seeking a mid-level QA Engineer to own quality for a platform integrating with multiple AI providers. You will blend manual testing, test automation, and AI-specific validation to ensure enriched outputs are accurate and reliable in production.

The role covers designing manual test plans, building automated suites with Playwright/Cypress/Selenium, API testing, and collaborating with DevOps for CI/CD. Strong QA instincts and clear bug reporting are essential.

Qualifications

• 3-5 years of QA experience including manual and automation testing.
• Hands-on automation with Playwright, Cypress or Selenium.
• Strong API testing skills (Postman, REST/pytest).
• Experience testing AI-related or non-deterministic systems is a plus.

Responsabilités

• Design and execute manual test plans and exploratory testing for web and API features.
• Build and maintain automated test suites for UI, API, and integration layers and integrate with CI/CD.
• Validate AI-enriched outputs for accuracy, relevance, and consistency.
• Test integrations with multiple AI providers and handle timeouts, rate limits, and fallbacks.
• Log clear, reproducible bugs and collaborate with DevOps and product teams.

Connaissances

Manual testing

Automation testing

Playwright

Cypress

Selenium

API testing

Postman

Python/JavaScript

Bug reporting

Outils

Playwright

Cypress

Selenium

Postman

REST Assured

Description du poste

About the Role

We are looking for a mid-level QA Engineer to own quality for a platform that integrates with multiple AI providers and uses them for data enrichment. You will combine manual testing, test automation, and AI-specific validation to make sure enriched outputs are accurate, consistent, and reliable in production.

Responsibilities

• Design and execute manual test plans, test cases, and exploratory testing for web and API features.
• Build and maintain automated test suites for UI, API, and integration layers, and wire them into CI/CD.
• Validate AI-enriched outputs for accuracy, relevance, consistency, and hallucination risk, using golden datasets and evaluation criteria.
• Test integrations with multiple AI providers (such as OpenAI, Anthropic, and others), including failure handling, timeouts, rate limits, fallback behavior, and provider response variations.
• Compare output quality across providers and model versions, and flag regressions when models or prompts change.
• Test prompt changes, edge cases, adversarial inputs, and data quality issues in the enrichment pipeline.
• Monitor cost, latency, and token usage behavior in test scenarios and report anomalies.
• Log clear, reproducible bugs and work closely with developers, product owners, and DevOps to resolve them.
• Contribute to QA processes, documentation, and release sign-off.

Required Skills

• 3-5 years of QA experience covering both manual and automation testing.
• Hands-on experience with automation tools such as Playwright, Cypress, or Selenium.
• Strong API testing skills (Postman, REST Assured, or pytest/requests-based frameworks).
• Solid understanding of testing non-deterministic systems, including LLM output evaluation, prompt testing, and handling variability in results.
• Experience testing third-party integrations, webhooks, and asynchronous or queue-based workflows.
• Familiarity with CI/CD pipelines (GitHub Actions, GitLab CI, or Jenkins) and version control with Git.
• Working knowledge of SQL and basic scripting in Python or JavaScript/TypeScript.
• Strong written communication and bug reporting skills.

Nice to Have

• Experience with AI evaluation tools or frameworks (such as promptfoo, DeepEval, or Ragas).
• Understanding of embeddings, RAG pipelines, and vector databases.
• Experience with performance and load testing (k6, JMeter, Locust).
• Exposure to Rails or Python/FastAPI backends, Redis, and background job systems like Sidekiq.
• Familiarity with AWS environments and log/monitoring tools.
• Experience with data-heavy or healthcare-adjacent products, including data privacy awareness.

What Success Looks Like

Within three months, you will have a stable automated regression suite covering core enrichment flows, a repeatable AI output evaluation process, and clear quality metrics that the team trusts for release decisions.

What We Offer

• Work on a real production AI platform with multiple provider integrations.
• A collaborative engineering team and room to shape QA practices.
• Competitive salary and growth opportunities.