Testing- AI Test Lead Manager

EY; Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

EY is seeking a QE AI Test Manager in Bengaluru, India. In this role, you will lead a team focused on testing AI models and systems, ensuring data quality and model validation.

Your responsibilities will include defining test strategies, managing stakeholders, and establishing best practices in AI testing to deliver effective business outcomes.

Qualifications

• Strong understanding of AI/ML concepts and Agentic AI systems.
• Ability to define validation strategies for AI models and systems.
• Experience managing global stakeholders and client interactions.

Responsibilities

• Define and drive end-to-end test strategy and governance for AI systems.
• Lead and mentor QE teams delivering AI testing engagements.
• Oversee data validation, model evaluation, and anomaly detection.

Skills

Understanding of AI/ML concepts

Test automation tools expertise (Selenium, Playwright, PyTest)

Proficiency in Python

Experience in Agile environments

Strong analytical and problem-solving skills

Excellent communication skills

Tools

GitHub Copilot

Cloud platforms (Azure, AWS, GCP)

Job description

The Opportunity

As a QE AI Test Manager, you will lead a team of quality engineering professionals to define and execute testing strategies for AI models and systems, including Narrow AI, Generative AI, and Agentic AI. You will be responsible for ensuring data quality, model validation, and overall testing governance across the AI lifecycle. The role also involves driving AI testing capabilities, managing stakeholders, planning resource requirements, and enabling best practices across engagements. You will collaborate with global stakeholders and clients to deliver business outcomes while building scalable AI testing competency within the organization.

Your Key Responsibilities

• Define and drive end-to-end test strategy and governance for AI systems
• Lead and mentor QE teams delivering AI testing engagements
• Oversee data validation, model evaluation, anomaly detection, and statistical analysis
• Ensure AI systems meet functional, performance, ethical, and responsible AI requirements
• Establish metrics and KPIs to measure AI system effectiveness and quality
• Manage stakeholder communication across clients and internal teams
• Plan and forecast resource requirements for AI testing engagements
• Drive adoption of best practices, tools, and frameworks across AI testing delivery
• Identify risks in AI models and define mitigation strategies
• Prepare and present testing reports, risks, and progress updates to stakeholders
• Maintain documentation of testing processes, results, and learnings

Skills and attributes for success

• Strong understanding of AI/ML concepts, LLMs, and Agentic AI systems
• Ability to define validation strategies for AI models and systems
• Experience managing global stakeholders and client interactions
• Expertise in test automation tools (Selenium, Playwright, PyTest)
• Proficiency in Python programming
• Exposure to GitHub Copilot or similar AI-assisted development tools
• Strong understanding of cloud platforms (Azure, AWS, GCP)
• Experience working in Agile environments
• Strong analytical, problem-solving, and decision-making skills
• Excellent communication, leadership, and collaboration skills
• Ability to operate in fast-paced, dynamic environments