Quality Assurance Test Lead

Incedo Inc.; San Diego (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Incedo Inc. is seeking an experienced QA Automation Lead to drive automation testing across web and mobile platforms.

You will design scalable automation frameworks using Selenium with Java, TestNG, and Playwright, while integrating AI/ML evaluation and Generative AI testing into the QA strategy. The role requires 10–15 years in QA and test automation, strong OOP and Java skills, and experience with AI model evaluation.

Qualifications

• Bachelor's degree in CS/IT/Engineering or related field.
• 10–15 years of QA and test automation experience.
• Strong OOP and Java experience.
• Hands-on Playwright and TestNG experience.
• Experience with AI/ML evaluation and Generative AI testing.

Responsibilities

• Lead end-to-end automation testing for web and mobile apps.
• Design and maintain scalable automation frameworks.
• Plan AI/ML and Generative AI testing, including model evaluation and prompt testing.
• Create test plans, suites, and test cases (manual and automated).
• Collaborate with developers, PO, BA, and QA teams to enable Shift-Left testing.

Skills

Java

Selenium

Playwright

TestNG

Automation testing

AI testing

LLM testing

Education

Bachelor's degree in Computer Science/IT/Engineering

Tools

JIRA

SQL Server

Azure DevOps

Job description

Incedo Inc. is a high-growth Digital, Data and AI Transformation Specialist firm headquartered in New Jersey. We are a long-term strategy execution partner for Fortune 500 enterprises, operating at the intersection of business and technology across Banking & Payments, Wealth Management, Telecom, Hi-Tech, and Life Sciences. We are building Incedo 4.0 - an AI-native, execution-focused, founder-led organization designed for scale, speed, and long-term impact.

Incedo delivers ROI from AI @ Scale through the "Power of 3":

• Engineering & Operations excellence

About the Role

We are seeking an experienced and results-driven QA Automation Lead to join our team. The ideal

candidate will be responsible for leading automation testing initiatives, designing and maintaining scalable automation frameworks, and ensuring the delivery of high-quality web and mobile applications.

This role requires strong expertise in Java, Selenium, Playwright, TestNG, and Automation Testing frameworks, along with hands-on experience in AI model evaluation and Generative AI testing. The candidate should be able to assess LLM behaviour, evaluate prompts and responses, validate AI/ML outputs, and collaborate effectively with developers, business analysts, product owners, and stakeholders to drive quality across the software development lifecycle.

Key Responsibilities

• Lead and drive end-to-end automation testing efforts for web and mobile applications.
• Design, develop, maintain, and execute automated test scripts using Selenium with Java, TestNG Framework, and Playwright.
• Develop and maintain robust automation frameworks that support regression, functional, and integration testing.
• Create comprehensive test plans, test suites, test scenarios, and automated/manual test cases.
• Plan and execute AI/ML and Generative AI testing, including model evaluation, prompt testing, response validation, and behavior validation.
• Define and execute test scenarios for LLM-based applications, including accuracy, consistency, relevance, robustness, safety, and response quality.
• Evaluate AI model outputs against expected behavior and business requirements and document defects, anomalies, and model-quality findings.
• Create comprehensive test plans and test cases covering both traditional software functionality and AI/LLM behavior.
• Collaborate closely with Developers, Product Owners, Business Analysts, and QA teams to implement Shift-Left Testing practices.
• Participate actively in Agile ceremonies, including sprint planning, reviews, and retrospectives.
• Identify, document, track, and verify defects using JIRA or similar defect management tools.
• Leverage modern AI-powered testing and productivity tools to improve test case creation, automation development, defect analysis, documentation, and overall QA efficiency.
• Utilize Generative AI tools such as GitHub Copilot, ChatGPT, Microsoft Copilot, or similar platforms to accelerate test automation development and enhance testing practices while adhering to organizational standards and security guidelines.

Required Qualifications

• Bachelor's degree in computer science, Information Technology, Engineering, or a related field.
• 10-15 years of overall experience in Software Quality Assurance and Test Automation.
• Strong understanding of Object-Oriented Programming (OOP) concepts.
• Hands-on experience with Java programming.
• Strong experience with Playwright automation framework.
• Extensive experience with TestNG framework/Page Object Model (POM).
• Working knowledge of Microsoft SQL Server and database validation techniques.
• 2+ years of hands-on experience in AI model evaluation, Generative AI testing, or ML validation.
• Hands-on experience designing and executing test scenarios for LLM-based or Generative AI applications, including prompt/response validation and model output evaluation.
• Hands-on experience using AI-powered development, testing, or productivity tools such as GitHub Copilot, Microsoft Copilot, ChatGPT, Gemini, Cursor, or similar solutions to improve software testing and automation efficiency.
• Understanding of AI-assisted software development and testing best practices.
• Strong analytical, troubleshooting, and problem-solving capabilities.
• Excellent verbal and written communication skills.

Preferred Qualifications

• Experience with API Testing using tools/frameworks such as Postman, Rest Assured, or similar.
• Exposure to Mobile Application Testing (Android/iOS).
• Experience with Azure DevOps/TFS Integration.
• Familiarity with CI/CD tools and automated deployment pipelines.
• Experience with test reporting and quality metrics dashboards.
• Experience with AI evaluation frameworks, LLM observability/testing tools, or automated evaluation techniques.