Quality Assurance Test Lead

Incedo Inc.; Charlotte (NC); On-site; USD 120,000 - 170,000; Full time

Job summary

Incedo Inc. is seeking an experienced QA Automation Lead to join our team. You will lead automation testing initiatives for web and mobile applications, designing scalable frameworks and ensuring delivery of high-quality software.

The role requires strong Java expertise, Selenium, Playwright, and TestNG, plus hands-on AI/Generative AI testing, including model evaluation and prompt testing. You will collaborate across engineering, analytics, product, and QA to advance quality across the SDLC.

Qualifications

• Bachelor’s degree in computer science, IT, engineering, or related field.
• 10–15 years in software QA and test automation.
• Strong OOP concepts and hands-on Java experience.
• Extensive Playwright and TestNG experience; POM.
• SQL Server and database validation knowledge.
• 2+ years AI model evaluation or Generative AI testing experience.

Responsibilities

• Lead end-to-end automation testing for web and mobile apps.
• Design and maintain scalable automation frameworks.
• Develop test plans, suites, scenarios, and test cases.
• Plan AI/ML and Generative AI testing, including model evaluation.
• Validate LLM prompts and responses against requirements.
• Collaborate with Developers, Product Owners, BAs, and QA teams; promote Shift-Left testing.
• Track defects using JIRA and verify fixes.

Skills

Analytical thinking

Troubleshooting

Problem solving

Verbal communication

Written communication

Education

Bachelor's degree in CS/IT/Engineering

Tools

Selenium with Java

TestNG

Playwright

SQL Server

Job description

Incedo Inc. is a high-growth Digital, Data and AI Transformation Specialist firm headquartered in New Jersey. We are a long-term strategy execution partner for Fortune 500 enterprises, operating at the intersection of business and technology across Banking & Payments, Wealth Management, Telecom, Hi-Tech, and Life Sciences. We are building Incedo 4.0 - an AI-native, execution-focused, founder-led organization designed for scale, speed, and long-term impact.

Incedo delivers ROI from AI @ Scale through the “Power of 3”:

• Engineering & Operations excellence

About the Role

We are seeking an experienced and results-driven QA Automation Lead to join our team. The ideal

candidate will be responsible for leading automation testing initiatives, designing and maintaining scalable automation frameworks, and ensuring the delivery of high-quality web and mobile applications.

This role requires strong expertise in Java, Selenium, Playwright, TestNG, and Automation Testing frameworks, along with hands-on experience in AI model evaluation and Generative AI testing. The candidate should be able to assess LLM behaviour, evaluate prompts and responses, validate AI/ML outputs, and collaborate effectively with developers, business analysts, product owners, and stakeholders to drive quality across the software development lifecycle.

Key Responsibilities

• Lead and drive end-to-end automation testing efforts for web and mobile applications.
• Design, develop, maintain, and execute automated test scripts using Selenium with Java, TestNG Framework, and Playwright.
• Develop and maintain robust automation frameworks that support regression, functional, and integration testing.
• Create comprehensive test plans, test suites, test scenarios, and automated/manual test cases.
• Plan and execute AI/ML and Generative AI testing, including model evaluation, prompt testing, response validation, and behavior validation.
• Define and execute test scenarios for LLM-based applications, including accuracy, consistency, relevance, robustness, safety, and response quality.
• Evaluate AI model outputs against expected behavior and business requirements and document defects, anomalies, and model-quality findings.
• Create comprehensive test plans and test cases covering both traditional software functionality and AI/LLM behavior.
• Collaborate closely with Developers, Product Owners, Business Analysts, and QA teams to implement Shift-Left Testing practices.
• Participate actively in Agile ceremonies, including sprint planning, reviews, and retrospectives.
• Identify, document, track, and verify defects using JIRA or similar defect management tools.
• Leverage modern AI-powered testing and productivity tools to improve test case creation, automation development, defect analysis, documentation, and overall QA efficiency.
• Utilize Generative AI tools such as GitHub Copilot, ChatGPT, Microsoft Copilot, or similar platforms to accelerate test automation development and enhance testing practices while adhering to organizational standards and security guidelines.

Required Qualifications

• Bachelor’s degree in computer science, Information Technology, Engineering, or a related field.
• 10–15 years of overall experience in Software Quality Assurance and Test Automation.
• Strong understanding of Object-Oriented Programming (OOP) concepts.
• Hands-on experience with Java programming.
• Strong experience with Playwright automation framework.
• Extensive experience with TestNG framework/Page Object Model (POM).
• Working knowledge of Microsoft SQL Server and database validation techniques.
• 2+ years of hands-on experience in AI model evaluation, Generative AI testing, or ML validation.
• Hands-on experience designing and executing test scenarios for LLM-based or Generative AI applications, including prompt/response validation and model output evaluation.
• Hands-on experience using AI-powered development, testing, or productivity tools such as GitHub Copilot, Microsoft Copilot, ChatGPT, Gemini, Cursor, or similar solutions to improve software testing and automation efficiency.
• Understanding of AI-assisted software development and testing best practices.
• Strong analytical, troubleshooting, and problem-solving capabilities.
• Excellent verbal and written communication skills.

Preferred Qualifications

• Experience with API Testing using tools/frameworks such as Postman, Rest Assured, or similar.
• Exposure to Mobile Application Testing (Android/iOS).
• Experience with Azure DevOps/TFS Integration.
• Familiarity with CI/CD tools and automated deployment pipelines.
• Experience with test reporting and quality metrics dashboards.
• Experience with AI evaluation frameworks, LLM observability/testing tools, or automated evaluation techniques.