Test Lead – CT Engineering, Product Testing

Jobtailor; Bengaluru; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

Jobtailor in Bengaluru is seeking a senior QA Automation Architect to lead test activities, set timelines, and supervise a team of testers across AI and traditional domains.

You will define test strategies, drive automation adoption, integrate with CI/CD, and mentor cross-functional groups while ensuring quality and risk mitigation for complex AI-driven products.

Qualifications

• Hands-on experience building and adopting large-scale automation frameworks across UI, functional, API and back-end integration.
• Experience in Selenium with Java / Python - scripting/coding level.
• Proficiency in testing concepts/standards & Agile methodology.
• Knowledge of Azure DevOps or any relevant defect management tool and CI/CD.
• Experience and knowledge of AI testing concepts and model evaluation.

Responsibilities

• Lead the test activities to ensure product quality, setting project timelines and supervising the activities of other testers.
• Drive the identification of the test scope and test environment set-up, prepare test plans, identify relevant metrics and keep track of required effort.
• Manage the execution of tests, review reports and interpret recommendations for product or design changes.
• Define and implement comprehensive test strategies, standards, and risk-based coverage.
• Define and implement automation strategies across functional and AI testing domains, ensuring scalability and maintainability.
• Evaluate and adopt cutting-edge automation tools and technologies, including AI-assisted test generation and validation.

Skills

Automation testing

AI testing

Selenium

CI/CD

Azure DevOps

Scripting (Python/Java)

Leadership

Education

Bachelor's degree in CS/IT/Engineering

Tools

Selenium WebDriver

Python

Java

Job description

Responsibilities

• Lead the test activities to ensure product quality, setting project timelines and supervising the activities of other testers.
• Drive the identification of the test scope and test environment set-up, prepare test plans, identify relevant metrics and keep track of required effort.
• Manage the execution of tests, review reports and interpret recommendations for product or design changes.
• Define and implement comprehensive test strategies, standards, and risk-based coverage.
• Define and implement automation strategies across functional and AI testing domains, ensuring scalability and maintainability.
• Evaluate and adopt cutting-edge automation tools and technologies, including AI-assisted test generation and validation.
• Drive automation coverage goals, balancing risk-based testing with efficiency improvements to reduce manual effort.
• Architect and maintain reusable automation frameworks for both traditional and AI-driven applications, integrated with CI/CD pipelines.
• Define and implement automation strategies across functional and AI testing domains, ensuring scalability and maintainability.
• Design, develop, and maintain standard and ad-hoc reports tailored to business needs.
• Implement automated reporting processes to reduce manual effort and improve timeliness.
• Drive defect triage, prioritization, and release readiness decisions.
• Develop relationships with other engineering groups to evaluate, recommend and implement testing procedures for products, systems, components or modifications.
• Implement AI testing strategies including DeepEval for model evaluation, ensuring robustness and reliability of outputs.
• Design and execute test plans for RAG (Retrieval-Augmented Generation) frameworks, validating retrieval and generation accuracy.
• Develop agentic validation workflows for multi-agent systems, focusing on reasoning accuracy and orchestration.
• Conduct bias, fairness, and ethical compliance testing for AI models.
• Build reusable automation frameworks for AI testing integrated with CI/CD pipelines.
• Lead and mentor a cross-functional testing team, fostering collaboration across AI, data engineering, and product development groups.
• Define strategic testing roadmaps for AI-driven products and manage stakeholder communication on quality metrics and risks.

Requirements

• Bachelor's degree in Computer Science, Engineering, IT, or related field and 12+ years of relevant work experience.
• Should be able to implement automation strategy and guide the team.
• Proficiency in testing concepts/standards & Agile methodology.
• Good understanding of Automation Testing concepts and ability to implement automation/performance strategy.
• Hands‑on experience authoring, building and adopting large‑scale automation frameworks across UI, functional, API and back‑end integration.
• Experience in Selenium with Java / Python - scripting/coding level.
• Excellent analytical and problem‑solving skills.
• Experience and knowledge of Azure DevOps or any relevant defect management tool and CI/CD.
• Self‑starter, with flexibility to adjust to multiple demands, shifting priorities, ambiguity and rapid change.
• Excellent written and verbal communication skills.
• Ability to understand and integrate with cultural differences and engage with virtual cross‑cultural, cross‑border teams.
• Experience in supporting small to medium size projects and the relevant Performance indicators and metrics.

Core Competencies

Demonstrates expertise in defining and implementing automation strategies for functional and AI testing, with a strong focus on building scalable automation frameworks integrated with CI/CD pipelines. Proven ability to lead cross‑functional teams, manage testing activities, and ensure product quality through effective communication and stakeholder engagement.