Quality Assurance Engineer - AI Chatbot

Hyqoo; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Hyqoo is seeking a QA Engineer in the United States with 5+ years of manual testing experience in chatbots and conversational AI. You will validate AI responses, test flows, and ensure a reliable user experience across integrations with knowledge bases and CRM systems.

You will design and execute test cases, perform API and SQL validation, log defects in Jira, and contribute to regression testing. Some automation awareness with Selenium/Playwright is a plus.

Qualifications

• 5+ years of hands-on manual testing experience.
• Experience testing chatbots, conversational AI, or virtual assistants.
• Strong understanding of test case design, test execution, defect reporting, and regression testing.
• Hands‑on experience with Jira for defect tracking and test management.
• Experience with Postman and API testing.
• Working knowledge of SQL for database validation.
• Ability to evaluate AI responses and identify hallucinations, irrelevant answers, and inconsistent outputs.
• Understanding of conversational flows, context retention, and edge‑case testing.
• Strong analytical, problem‑solving, and documentation skills.

Responsibilities

• Perform manual testing of chatbots, conversational AI applications, and virtual assistants.
• Validate AI-generated responses for accuracy, relevance, consistency, and contextual understanding.
• Test conversational flows, context retention, multi-turn conversations, and edge cases.
• Identify AI hallucinations, incorrect responses, unexpected behavior, and chatbot failures.
• Test chatbot integrations with knowledge bases, CRM systems, and backend services.
• Perform API testing using Postman to validate chatbot integrations and backend responses.
• Execute SQL queries to validate data accuracy and backend records.
• Test fallback mechanisms, error handling, and escalation to live customer-service agents.
• Create, maintain, and execute test cases, test scenarios, and test documentation.
• Log, track, and manage defects using Jira and collaborate with development teams for resolution.
• Perform regression testing to ensure existing chatbot functionality remains stable after updates.
• Support automation testing using Selenium or Playwright, where applicable.

Skills

Manual Testing

Chatbot Testing

AI Output Evaluation

Convo Flows

Regression Testing

Test Case Design

Documentation

Jira

Postman

API Testing

SQL

Tools

Jira

Postman

SQL

Job description

We are looking for a QA Engineer with 5+ years of hands-on manual testing experience, preferably in chatbots, conversational AI, or virtual assistants. The ideal candidate will be responsible for validating chatbot functionality, AI-generated responses, conversational flows, API integrations, and overall response quality to ensure a reliable user experience.

Key Responsibilities

• Perform manual testing of chatbots, conversational AI applications, and virtual assistants.
• Validate AI-generated responses for accuracy, relevance, consistency, and contextual understanding.
• Test conversational flows, context retention, multi-turn conversations, and edge cases.
• Identify AI hallucinations, incorrect responses, unexpected behavior, and chatbot failures.
• Test chatbot integrations with knowledge bases, CRM systems, and backend services.
• Perform API testing using Postman to validate chatbot integrations and backend responses.
• Execute SQL queries to validate data accuracy and backend records.
• Test fallback mechanisms, error handling, and escalation to live customer-service agents.
• Create, maintain, and execute test cases, test scenarios, and test documentation.
• Log, track, and manage defects using Jira and collaborate with development teams for resolution.
• Perform regression testing to ensure existing chatbot functionality remains stable after updates.
• Support automation testing using Selenium or Playwright, where applicable.

Required Skills & Qualifications

• 5+ years of hands-on manual testing experience.
• Experience testing chatbots, conversational AI, or virtual assistants.
• Strong understanding of test case design, test execution, defect reporting, and regression testing.
• Hands‑on experience with Jira for defect tracking and test management.
• Experience with Postman and API testing.
• Working knowledge of SQL for database validation.
• Ability to evaluate AI responses and identify hallucinations, irrelevant answers, and inconsistent outputs.
• Understanding of conversational flows, context retention, and edge‑case testing.
• Strong analytical, problem‑solving, and documentation skills.

Required Technical Skills

• Manual Testing
• Chatbot & Conversational AI Testing
• Jira
• Postman
• API Testing
• SQL

Preferred Skills

• Experience with Selenium or Playwright for test automation.
• Exposure to AI‑powered applications, LLM‑based chatbots, or generative AI solutions.
• Experience testing chatbot integrations with knowledge bases, CRM platforms, and customer‑support systems.
• Familiarity with API automation and regression testing frameworks.