Senior AI QA Engineer

Biz2X; Dadri; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Biz2X is seeking an AI QA Engineer to join our Quality Engineering team in India. You will test GenAI and AI-powered applications, build and maintain Playwright/Selenium automation, and validate AI conversational workflows.

You will collaborate with Product, Engineering, Data Science, and AI teams to ensure quality, reliability, and accuracy of next-generation AI-powered products, including prompt testing and grounding validation.

Qualifications

• Bachelor's degree in Engineering (B.E./B.Tech) in CS/IT/electronics or related fields.
• Strong hands-on experience with Playwright (Mandatory).
• Experience with Selenium Automation.
• Experience testing GenAI, LLM, Chatbot, or AI-powered applications.
• Hands-on experience with RAG Testing.
• Familiar with hallucination, grounding, and response validation.

Responsibilities

• Design, develop, and maintain test automation frameworks using Playwright and Selenium.
• Perform end-to-end testing of web applications and AI-powered UX.
• Validate chatbot conversations, AI-generated responses, and workflows.
• Execute testing for GenAI, LLM, and RAG applications.
• Perform prompt testing, grounding validation, and hallucination detection.
• Validate AI agent workflows, multi-step reasoning, and tool integrations.
• Execute functional, regression, integration, API, database, and UAT testing.
• Perform SQL-based backend validations and data quality checks.
• Collaborate with cross-functional teams to define quality strategies.
• Participate in Agile ceremonies and contribute to quality improvements.

Skills

Playwright

Selenium Automation

GenAI Testing

LLM Testing

RAG Testing

AI Testing

Chatbot Testing

Prompt Testing

Agentic AI

Python

Java

API Testing

Postman

SQL

AWS

CI/CD

Jira

Test Automation

Quality Assurance

Education

B.E./B.Tech in Computer Science / IT / Electronics

Tools

Pinecone

ChromaDB

Weaviate

FAISS

AWS

Bedrock / Azure OpenAI

Job description

Employment Type: Full-Time

About the Role

We are looking for a highly skilled AI QA Engineer to join our Quality Engineering team. This role combines traditional automation testing expertise with modern AI application validation. The ideal candidate will have hands‑on experience testing GenAI applications, LLM-powered systems, chatbots, RAG applications, and AI-driven workflows, along with strong automation experience using Playwright.

You will work closely with Product, Engineering, Data Science, and AI teams to ensure the quality, reliability, and accuracy of next‑generation AI‑powered products.

Key Responsibilities

• Design, develop, and maintain test automation frameworks using Playwright and Selenium.
• Perform end‑to‑end testing of web applications and AI‑powered user experiences.
• Validate chatbot conversations, AI‑generated responses, and conversational workflows.
• Execute testing for GenAI, LLM, and RAG (Retrieval‑Augmented Generation) applications.
• Perform prompt testing, grounding validation, and hallucination detection.
• Validate AI agent workflows, multi‑step reasoning, and tool integrations.
• Execute functional, regression, integration, API, database, and UAT testing.
• Perform SQL‑based backend validations and data quality checks.
• Collaborate with cross‑functional teams to define and implement quality strategies.
• Participate in Agile ceremonies and contribute to continuous quality improvement.

Must‑Have Qualifications

• Bachelor's degree in Engineering (B.E./B.Tech) in Computer Science, Information Technology, Electronics, or related fields.
• Strong hands‑on experience with Playwright (Mandatory).
• Experience with Selenium Automation.
• Experience testing GenAI, LLM, Chatbot, or AI‑powered applications.
• Hands‑on experience with RAG Testing.
• Strong understanding of:
• Hallucination Testing
• Response Validation
• Grounding Validation
• Experience with API Testing using Postman or REST APIs.
• Strong SQL skills and experience with databases such as:
• MySQL
• MongoDB
• DynamoDB
• Experience with Python or Java.
• Familiarity with:
• Git
• Jira
• Agile Methodologies

Good‑to‑Have

• Agentic AI Testing
• MCP (Model Context Protocol)
• Exposure to LangSmith, Ragas, Promptfoo, DeepEval
• Experience with vector databases:
• Pinecone
• ChromaDB
• Weaviate
• FAISS
• AWS or GCP experience
• Exposure to Amazon Bedrock or Azure OpenAI

Preferred Project Experience

Candidates with experience in any of the following will be highly preferred:

GenAI & Chatbot Testing

• Conversational AI testing

RAG Application Testing

• Retrieval quality validation
• Embedding and chunking validation
• Citation accuracy testing

Agentic AI Testing

• AI workflow orchestration
• Tool and API integrations

Skills

Playwright, Selenium, Generative AI, LLM Testing, RAG Testing, AI Testing, Chatbot Testing, Prompt Testing, Agentic AI, Python, Java, API Testing, Postman, SQL, AWS, CI/CD, Jira, Test Automation, Quality Assurance

Why Join Us?

Join a team building innovative AI‑powered products where you'll play a critical role in ensuring reliability, accuracy, and exceptional user experiences through advanced AI quality engineering practices.