Test Engineer

Quantiphi Analytics Solutions; Mumbai, Bengaluru; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

Quantiphi Analytics Solutions in Mumbai/Bangalore Hybrid is seeking a Senior Test Engineer - AI/ML to own end-to-end QA for AI-enabled applications and automate UI/API testing. You will design reusable automation frameworks, validate AI model outputs, and accelerate testing with GenAI tools.

The role emphasizes collaboration with ML and software teams, strong debugging skills, and mentoring junior engineers in automation-first QA practices.

Qualifications

• Strong STLC/SDLC, Agile, and Scrum experience.
• Hands-on in designing test strategies, scenarios, and execution.
• UI automation with Playwright and API automation with REST Assured (Java) or Python Requests.

Responsibilities

• Design, develop, and maintain UI and API automation test suites using modern automation frameworks.
• Own end-to-end testing activities including test planning, automation development, execution, defect management, and reporting.
• Validate AI/ML applications, Generative AI solutions, LLM-powered applications, and intelligent workflows.
• Design and execute functional, integration, regression, and end-to-end test automation for traditional and AI-enabled apps.
• Develop reusable automation frameworks, libraries, and utilities.
• Automate API testing using REST Assured or Python Requests; develop UI automation with Playwright.
• Validate backend data using SQL and database verification; ensure data integrity.
• Develop automated validations for AI model outputs, prompt responses, RAG pipelines, and chatbot conversations.
• Validate AI responses for accuracy, relevance, grounding, safety, and bias; ensure quality gates in CI/CD.
• Collaborate with ML Engineers, Software Engineers, and Product Owners to define AI quality validation strategies.
• Mentor junior engineers and promote automation-first and AI-first Quality Engineering practices.

Skills

UI automation

API automation

Playwright

REST Assured

Python Requests

SQL

CI/CD

GenAI testing

LLMs/RAG

Troubleshooting

Mentoring

Tools

Playwright

REST Assured

Python

Postman

Jenkins

GitHub Actions

Git

Job description

Role: Senior Test Engineer - AI/ML

Experience: 3-6 years

Location: Mumbai/Bangalore (Hybrid)

Brief About Role

We are looking for a Senior Test Automation Engineer to join our Quality Engineering (TaaS) team. The ideal candidate should possess strong expertise in modern Quality Engineering practices, UI and API automation, CI/CD integration, and automation framework development. The role requires an engineer who can independently own end-to-end testing activities for enterprise applications while contributing to reusable automation frameworks and AI-powered testing accelerators.

The candidate should be passionate about quality, automation, continuous improvement, and leveraging GenAI to improve testing productivity.

Key Responsibilities

• Design, develop, and maintain UI and API automation test suites using modern automation frameworks.
• Independently own end-to-end testing activities including test planning, automation development, execution, defect management, and reporting.
• Validate AI/ML applications, Generative AI solutions, LLM-powered applications, and intelligent workflows.
• Design and execute functional, integration, regression, and end-to-end test automation for traditional and AI-enabled applications.
• Develop and maintain reusable automation frameworks, libraries, and utilities.
• Automate API testing using REST Assured (Java) or Python Requests.
• Develop UI automation using Playwright.
• Validate backend data using SQL and perform database verification.
• Develop automated validations for AI model outputs, prompt responses, RAG pipelines, and chatbot conversations.
• Validate AI responses for correctness, consistency, relevance, grounding, safety, hallucinations, and bias.
• Collaborate with ML Engineers, Software Engineers, and Product Owners to define AI quality validation strategies.
• Integrate automated test suites into CI/CD pipelines and implement quality gates.
• Leverage GenAI tools to accelerate test case generation, automation development, debugging, documentation, and productivity.
• Contribute to AI testing frameworks, accelerators, and Quality Engineering best practices.
• Mentor junior engineers and promote automation-first and AI-first Quality Engineering practices.

Must have skills

• Strong understanding of Software Testing Life Cycle (STLC), SDLC, Agile, and Scrum methodologies.
• Hands-on experience in designing test strategies, test scenarios, test cases, and test execution.
• Strong expertise in UI automation using Playwright and API automation using REST Assured (Java) or Python Requests.
• Hands-on experience with Postman for API testing and validation.
• Strong SQL skills with hands-on experience in database testing and backend data validation.
• Experience integrating automation suites with CI/CD pipelines using Jenkins, GitHub Actions, or similar.
• Proficiency with Git and version control best practices.
• Hands-on experience testing AI/ML or Generative AI applications.
• Understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), chatbots, copilots, or AI agents.
• Experience validating AI responses for accuracy, relevance, consistency, hallucinations, toxicity, bias, and safety.
• Experience designing test scenarios for prompt engineering and AI workflow validation.
• Understanding of AI evaluation techniques and automated output validation.
• Familiarity with AI evaluation frameworks or libraries (e.g., DeepEval, Ragas, LangSmith, Promptfoo, MLflow, or similar).
• Experience using GenAI tools such as ChatGPT, GitHub Copilot, Kiro, Claude, or similar to accelerate test design, automation development, debugging, test data generation, and documentation.
• Strong debugging, root cause analysis, and analytical problem-solving skills.
• Ability to independently own end-to-end testing activities with minimal supervision.
• Excellent communication, collaboration, and stakeholder management skills.
• Ability to mentor junior engineers and promote automation and Quality Engineering best practices.
• Strong attention to detail with a quality-first and automation-first mindset.

Nice to have skills

• Knowledge of AWS services such as Amazon Bedrock, SageMaker, EC2, Lambda, API Gateway, S3, RDS, DynamoDB, CloudWatch, ECS/EKS, and IAM.
• Experience testing cloud-native applications deployed on AWS.
• Experience in Data Warehouse, ETL, or Data Pipeline testing.
• Experience with performance testing tools such as JMeter or k6.
• Exposure to Docker, Kubernetes, and containerized application testing.
• Relevant certifications in Playwright, AWS, Quality Engineering, or Agile are an added advantage.