QA Manual Engineer (AI Applications Testing) - Remote India
Position: Senior QA Manual Engineer - AI Applications
Experience: 7+ Years
Location: Remote (India)
Employment Type: Full-Time
About The Role
We are seeking a highly skilled QA Manual Engineer with experience testing AI-powered applications. The ideal candidate will have strong expertise in manual testing practices combined with hands-on experience validating Generative AI, Machine Learning, NLP, Chatbots, LLM-based applications, and AI-driven workflows.
Key Responsibilities
- Analyze business requirements, technical specifications, and AI solution requirements.
- Design and execute comprehensive test plans, test cases, and test scenarios for AI-driven applications.
- Perform Functional, Regression, Integration, System, Smoke, Sanity, and UAT testing.
- Validate AI model outputs for accuracy, relevance, consistency, bias, hallucinations, and performance.
- Test AI-powered features including Chatbots, Virtual Assistants, Recommendation Systems, and Document Intelligence solutions.
- Execute prompt testing and prompt validation for Generative AI applications.
- Conduct API testing using Postman and validate AI service integrations.
- Perform database validation and backend testing using SQL.
- Log, track, and manage defects using Jira, Azure DevOps, or similar tools.
- Collaborate with Product Managers, Data Scientists, AI Engineers, and Developers throughout the SDLC.
- Participate in Agile/Scrum ceremonies and support release validation activities.
- Create detailed QA reports, test metrics, and quality assessments.
Required Skills
- 7+ years of experience in Manual Testing.
- Strong understanding of SDLC, STLC, and Agile methodologies.
- Experience testing AI/ML-based applications.
- Hands-on experience validating Large Language Models (LLMs) such as GPT, Claude, Gemini, or similar AI systems.
- Experience testing Chatbots, Conversational AI, or NLP-based applications.
- Strong knowledge of prompt engineering and AI output validation.
- Experience with API testing using Postman.
- Strong SQL skills for data validation.
- Expertise in defect management using Jira or Azure DevOps.
- Strong analytical, troubleshooting, and communication skills.
Preferred Skills
- Experience with OpenAI, Azure OpenAI, Anthropic Claude, Gemini, or similar AI platforms.
- Understanding of AI evaluation metrics such as Accuracy, Precision, Recall, F1 Score, Hallucination Detection, and Bias Testing.
- Knowledge of AI Security Testing and Responsible AI principles.
- Exposure to Automation Testing tools such as Playwright, Selenium, or Cypress.
- Experience with AWS or Azure Cloud environments.
- Familiarity with Accessibility Testing (WCAG, NVDA, JAWS).
Nice to Have
- Experience testing RAG (Retrieval-Augmented Generation) solutions.
- Experience with AI Agents, Copilot solutions, or Agentic AI workflows.
- ISTQB Certification.
- Experience in SaaS, Healthcare, FinTech, or Enterprise products.
Qualifications
- Bachelor's Degree in Computer Science, Information Technology, or a related field.
- Relevant QA or AI certifications are a plus