DE-NCLC-SM-Agenetic AI

EY; Tada; On-site; INR 2,500,000 - 5,000,000; Full time

Job summary

EY in India is seeking a Senior Manager to strengthen the Agentic AI testing effort. You will own end-to-end test strategy, build automated regression and integration tests using Python-based tools, and drive release readiness.

You will validate AI behavior beyond UI/API, design test scenarios, maintain evaluation datasets, and ensure traceability of agent execution logs. The role requires collaboration with SMEs, Security and Compliance, and contributing to performance testing and UAT evidence.

Qualifications

• Experience with end-to-end test strategy and release readiness.
• Strong automation skills using Playwright, Cypress and Selenium.
• Experience validating AI-generated outputs and logs.

Responsibilities

• Own the end-to-end test strategy, test plans, acceptance-criteria coverage, and release-quality evidence for the Agentic product.
• Build and maintain automated regression and integration tests using Playwright, Cypress, Selenium, API testing frameworks, and Python testing libraries.
• Validate agent behavior beyond conventional UI/API testing, including tool selection and sequencing, function-call inputs/outputs, orchestration paths, refusal/escalation behavior, and human-in-the-loop controls.
• Design and execute positive, negative, ambiguous, adversarial, and out-of-scope scenarios for every relevant acceptance criterion.
• Maintain evaluation datasets and regression suites for AI-generated outputs, with focus on correctness, consistency, factual grounding, and prevention of regressions when prompts, models, configurations, or tools change.
• Validate agent execution logs and traceability across perception, reasoning, planning, execution, retries/corrections, and resulting outputs so test evidence can be reviewed by SMEs.
• Test security and access-control scenarios, including negative authorization cases and agent/tool boundaries, and coordinate with Security and Compliance SMEs on identified issues.
• Perform defect triage and root-cause analysis with developers, track defects through closure, and provide clear quality and release-readiness reporting through automated scripts/dashboards where practical.
• Support performance and regression testing and contribute evidence required for system testing, UAT, and release gates.

Skills

Python

Automation

Test strategy

API testing

Regression testing

Tools

Playwright

Cypress

Selenium

Job description

Job Description:

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.

Job Description: Senior Manager

Senior Manager stronger Python, automation, agentic AI

• Own the end-to-end test strategy, test plans, acceptance-criteria coverage, and release-quality evidence for the Agentic product.
• Build and maintain automated regression and integration tests using UHG provided tools such as Playwright, Cypress, Selenium, API testing frameworks, and Python testing libraries.
• Validate agent behavior beyond conventional UI/API testing, including tool selection and sequencing, function-call inputs/outputs, orchestration paths, refusal/escalation behavior, and human-in-the-loop controls.
• Design and execute positive, negative, ambiguous, adversarial, and out-of-scope scenarios for every relevant acceptance criterion.
• Maintain evaluation datasets and regression suites for AI-generated outputs, with focus on correctness, consistency, factual grounding, and prevention of regressions when prompts, models, configurations, or tools change.
• Validate agent execution logs and traceability across perception, reasoning, planning, execution, retries/corrections, and resulting outputs so test evidence can be reviewed by SMEs.
• Test security and access-control scenarios, including negative authorization cases and agent/tool boundaries, and coordinate with Security and Compliance SMEs on identified issues.
• Perform defect triage and root-cause analysis with developers, track defects through closure, and provide clear quality and release-readiness reporting through automated scripts/dashboards where practical.
• Support performance and regression testing and contribute evidence required for system testing, UAT, and release gates.

EY | Building a better working world

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.

Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.

Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.