Quality Assurance Engineer (Chatbot testing, Golden dataset)

AVENSYS CONSULTING PTE. LTD.; Singapore; On-site; SGD 60,000 - 90,000; Part time

Job summary

AVENSYS CONSULTING PTE. LTD. is seeking a QA Engineer with 3–4 years of experience in AI application testing to support testing activities for AI-powered solutions within the banking and financial services domain.

The role focuses on validating AI chatbots, document-processing solutions, and AI-generated responses using structured test data and quality benchmarks.

Qualifications

• 3–4 years of experience in software QA/testing, with hands-on experience in AI application testing.
• Experience testing AI/ML-based applications, chatbots, or GenAI solutions.
• Understanding of golden datasets / ground-truth datasets and their use in AI evaluation.
• Experience in validating AI outputs against expected results.

Responsibilities

• Perform functional and quality testing of AI-powered applications, including chatbots and document-processing solutions.
• Design, execute, and maintain test cases for AI/ML applications.
• Validate AI-generated responses for accuracy, relevance, consistency, and completeness.
• Work with golden datasets and other curated test data to evaluate AI model outputs against verified expected results.
• Identify and document defects, inconsistencies, hallucinations, and unexpected AI behaviour.
• Perform regression testing when AI models, prompts, datasets, or application features are updated.
• Test various chatbot scenarios, including different user queries, conversation flows, and edge cases.
• Validate AI applications that process and extract information from financial and business documents.
• Collaborate with developers, AI/ML engineers, business analysts, and other QA teams to investigate and resolve issues.
• Prepare test reports and provide clear feedback on AI application quality and test results.
• Ensure testing activities align with the quality and compliance expectations of the banking/financial services domain.

Skills

AI application testing

QA testing

AI/ML testing

Chatbot testing

Tools

API testing

Job description

Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success we have evolved to become one of the top trusted providers in Singapore and service a client base across banking and financial services, insurance, information technology, healthcare, retail, and supply chain.

We are currently looking to hire QA Engineer (AI application testing, Chatbots, golden datasets)

This is an exciting opportunity to expand your skill set, achieve job satisfaction and work-life balance. More details as below.

Job Type: One-year Renewable Contract

Job Summary

We are looking for a QA Engineer with 3–4 years of experience in AI application testing to support testing activities for AI-powered solutions within the banking and financial services domain.

The role will focus on validating AI chatbots, document-processing solutions, and AI-generated responses using structured test data and quality benchmarks.

Key Responsibilities

• Perform functional and quality testing of AI-powered applications, including chatbots and document-processing solutions.
• Design, execute, and maintain test cases for AI/ML applications.
• Validate AI-generated responses for accuracy, relevance, consistency, and completeness.
• Work with golden datasets and other curated test data to evaluate AI model outputs against verified expected results.
• Identify and document defects, inconsistencies, hallucinations, and unexpected AI behaviour.
• Perform regression testing when AI models, prompts, datasets, or application features are updated.
• Test various chatbot scenarios, including different user queries, conversation flows, and edge cases.
• Validate AI applications that process and extract information from financial and business documents.
• Collaborate with developers, AI/ML engineers, business analysts, and other QA teams to investigate and resolve issues.
• Prepare test reports and provide clear feedback on AI application quality and test results.
• Ensure testing activities align with the quality and compliance expectations of the banking/financial services domain.

Required Skills & Experience

• 3–4 years of experience in software QA/testing, with hands-on experience in AI application testing.
• Experience testing AI/ML-based applications, chatbots, or GenAI solutions.
• Understanding of golden datasets / ground-truth datasets and their use in AI evaluation.
• Experience in validating AI outputs against expected results.
• Strong understanding of test case design, defect management, regression testing, and QA processes.
• Experience testing document-processing or document-understanding applications is preferred.
• Good analytical and problem-solving skills.
• Ability to identify issues related to AI response quality, accuracy, and consistency.
• Experience in the banking or financial services domain will be an advantage.

Preferred

• Exposure to Generative AI / LLM testing.
• Knowledge of prompt testing and evaluation.
• Experience with AI evaluation metrics and test frameworks.
• Familiarity with API testing and automation tools.
• Experience working in Agile/Scrum environments.

WHAT’S ON OFFER

You will be remunerated with an excellent base salary and entitled to attractive company benefits. Additionally, you will get the opportunity to enjoy a fun and collaborative work environment, alongside a strong career progression.

Privacy Statement

Privacy Statement: Data collected will be used for recruitment purposes only. Personal data provided will be used strictly in accordance with the relevant data protection law and Avensys' privacy policy.