Quality Assurance Engineer

Knowledgesprint Technologies; Hyderabad; On-site; INR 800,000 - 1,400,000; Full time

Job summary

Knowledgesprint Technologies is seeking a QA engineer to perform comprehensive AI-focused testing in Hyderabad, India. You will validate LLM/GenAI/RAG workflows, assess tooling and data quality, and ensure robust tool selection, execution, and failure handling across diverse user scenarios.

The role emphasizes automation using Selenium/Playwright/Cypress, API testing via REST and Postman, and integration into CI/CD pipelines, with evaluation via AI quality tools like Ragas, DeepEval, Promptfoo

Qualifications

• Experience with test automation and API testing.
• Ability to craft AI-focused test scenarios and edge cases.
• Scripting in Python or similar languages for test automation.

Responsibilities

• Perform functional, regression, integration, and API testing for AI-powered applications.
• Test LLM, GenAI, RAG, and agentic AI workflows across diverse user scenarios.
• Evaluate LLM responses for accuracy, relevance, consistency, groundedness, and completeness.
• Validate RAG pipelines, including retrieval quality, context relevance, and response grounding.
• Test AI agents for tool selection, execution, workflow completion, and failure handling.
• Create AI test datasets, evaluation scenarios, edge cases, and adversarial test cases.
• Automate high-value test scenarios using Selenium, Playwright, Cypress, or similar frameworks.
• Perform API testing using REST APIs and Postman, and integrate automated tests into CI/CD pipelines.
• Use AI testing and evaluation tools such as Ragas, DeepEval, Promptfoo, and LangSmith for AI quality assessment.
• Analyze defects, perform root-cause analysis, validate fixes, and support performance, scalability, and reliability testing using Python or other scripting languages.

Tools

Selenium

Playwright

Cypress

Postman

REST APIs

Job description

Job Summary

Perform functional, regression, integration, and API testing for AI-powered applications. Test LLM, GenAI, RAG, and agentic AI workflows across diverse user scenarios. Evaluate LLM responses for accuracy, relevance, consistency, groundedness, and completeness. Validate RAG pipelines, including retrieval quality, context relevance, and response grounding. Test AI agents for tool selection, execution, workflow completion, and failure handling. Create AI test datasets, evaluation scenarios, edge cases, and adversarial test cases. Automate high-value test scenarios using Selenium, Playwright, Cypress, or similar frameworks. Perform API testing using REST APIs and Postman, and integrate automated tests into CI/CD pipelines. Use AI testing and evaluation tools such as Ragas, DeepEval, Promptfoo, and LangSmith for AI quality assessment. Analyze defects, perform root-cause analysis, validate fixes, and support performance, scalability, and reliability testing using Python or other scripting languages.

Responsibilities

• Perform functional, regression, integration, and API testing for AI-powered applications.
• Test LLM, GenAI, RAG, and agentic AI workflows across diverse user scenarios.
• Evaluate LLM responses for accuracy, relevance, consistency, groundedness, and completeness.
• Validate RAG pipelines, including retrieval quality, context relevance, and response grounding.
• Test AI agents for tool selection, execution, workflow completion, and failure handling.
• Create AI test datasets, evaluation scenarios, edge cases, and adversarial test cases.
• Automate high-value test scenarios using Selenium, Playwright, Cypress, or similar frameworks.
• Perform API testing using REST APIs and Postman, and integrate automated tests into CI/CD pipelines.
• Use AI testing and evaluation tools such as Ragas, DeepEval, Promptfoo, and LangSmith for AI quality assessment.
• Analyze defects, perform root-cause analysis, validate fixes, and support performance, scalability, and reliability testing using Python or other scripting languages.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.