Testing - AI Test Staff

EY; Bengaluru; On-site; INR 1,000,000 - 1,800,000; Full time

Job summary

EY in Bengaluru is hiring a QE AI Test Engineer to validate AI-powered solutions, including Narrow AI, Generative AI, and Agentic AI, across model validation, data quality, and performance metrics. You will design test scenarios, apply AI-specific testing methods, and collaborate with data scientists and developers in an Agile setting.

The role focuses on ensuring reliable, scalable, and trustworthy AI applications while leveraging Python-based automation tools and cloud platforms.

Qualifications

• Experience in testing AI models and systems.
• Solid understanding of software testing principles and quality engineering practices.
• Foundational knowledge of AI/ML concepts and model validation techniques.
• Familiarity with Large Language Models, Generative AI, and Agentic AI systems.

Responsibilities

• Validate AI models and systems across Narrow AI, Generative AI, and Agentic AI implementations to ensure quality, reliability, and performance.
• Design and execute test scenarios for AI-based applications using established testing principles.

Skills

AI Testing

Test Automation

Python

Selenium

Playwright

PyTest

LLMs & AI

Cloud Knowledge

Agile

Tools

Selenium

Playwright

PyTest

Job description

At EY, you’ll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we’re counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.

Service line - Position Title: Consulting – QE AI Test Engineer

At EY, we’re all in to shape your future with confidence.

We’ll help you succeed in a globally connected powerhouse of diverse teams and take your career wherever you want it to go.

Join EY and help to build a better working world.

About Global Delivery Services

Global Delivery Services refers to EY's worldwide network of service delivery centers. The GDS team plays an important role in EY’s strategy by ensuring effective support to EY’s growth agenda.

Our journey started in 2002 with approximately 200 people. Today we stand at 80,000+ professionals in ten locations around the world. We operate in Argentina, China, Hungary, India, Philippines, Poland, Sri Lanka, Mexico, Spain and the United Kingdom.

Client service is focused on providing Consulting, Assurance, Tax, Strategy & Transactions, and Knowledge support to our clients around the world. The teams enable account teams worldwide to provide seamless, high-quality, value-added support, helping deliver exceptional client service.

Enablement Services provides cost-effective, high-skilled, and innovative services to support EY’s global and local enablement teams. Markets, BMC, AWS, Finance and Accounting, Risk Management, Procurement, People Shared Services, IT Service Delivery and IT Global Infrastructure services, are among the services offered by Enablement Services.

Our innovation specialists serve the GDS Client Service and Enablement Services teams, along with Service Lines, Core Business Services and Sectors. The team brings the desired environment, technologies and skilled teams together for facilitation, rapid prototyping and innovative thinking. The competencies offered include analytics, digital, user experience, mobile technology, infrastructure, Microsoft technologies and open innovation.

The Opportunity

As a QE - AI Test Engineer, you will play a key role in validating and assuring the quality of AI-powered solutions, including Narrow AI, Generative AI, and Agentic AI systems. You will work closely with cross-functional teams to evaluate AI models, validate data quality, perform statistical analysis, and ensure AI applications meet functional, performance, and business requirements. This role provides an exciting opportunity to work at the forefront of AI innovation while applying quality engineering practices to build reliable, scalable, and trustworthy AI solutions.

Your Key Responsibilities

• Validate AI models and systems across Narrow AI, Generative AI, and Agentic AI implementations to ensure quality, reliability, and performance.
• Perform data validation activities, including data collection, data generation, data augmentation, exploratory data analysis, and assessment of data quality, bias, and privacy considerations.
• Evaluate AI model outputs against functional, business, and performance metrics to identify defects, inconsistencies, and improvement opportunities.
• Design and execute test scenarios for AI-based applications using established software testing principles and practices.
• Apply AI-specific testing approaches such as pairwise testing, metamorphic testing, back-to-back testing, bias testing, and drift testing.
• Utilize Python-based frameworks and automation tools to improve testing efficiency and coverage.
• Collaborate with developers, data scientists, product owners, and stakeholders to identify, track, and resolve quality issues throughout the development lifecycle.
• Support validation of Large Language Models (LLMs) and Agentic AI systems using established evaluation methodologies and frameworks.
• Contribute to continuous improvement initiatives by recommending enhancements to testing processes, methodologies, and quality standards.
• Work effectively within Agile teams to support delivery of high-quality AI solutions.

Skills And Attributes For Success

• Proven experience in testing AI models and systems
• Strong understanding of software testing principles, methodologies, and quality engineering practices.
• Foundational knowledge of AI/ML concepts, algorithms, and model validation techniques.
• Understanding of Large Language Models (LLMs), Generative AI, and Agentic AI systems.
• Experience with AI testing techniques such as pairwise testing, metamorphic testing, bias testing, drift testing, and back-to-back testing.
• Proficiency in Python programming for test development, validation, and automation activities.
• Experience with test automation tools such as Selenium, Playwright, PyTest, or similar frameworks.
• Knowledge of cloud technologies and platforms including Microsoft Azure, AWS, or Google Cloud Platform.
• Exposure to GitHub Copilot and AI-assisted development tools.
• Strong analytical, troubleshooting, and problem-solving capabilities.
• Excellent communication and stakeholder management skills.
• Ability to collaborate effectively in Agile delivery environments and multidisciplinary teams.

What We Offer You

At EY, we’ll develop you with future-focused skills and equip you with world‑class experiences. We’ll empower you in a flexible environment, and fuel you and your extraordinary talents in a diverse and inclusive culture of globally connected teams.

Learn more.

Are you ready to shape your future with confidence?

To help create an equitable and inclusive experience during the recruitment process, please inform us as soon as possible about any disability-related adjustments or accommodations you may need.

• Our Commitment: As a commitment, we persistently endeavour to embody our values, fulfil our purpose, and champion inclusiveness. Our dedication is to cultivate EY into an environment where diverse perspectives are celebrated, creating a supportive atmosphere for individuals to authentically be themselves and contribute their utmost.
• Professional Development: From entry-level employees to senior leaders, we believe in continuous learning. We offer opportunities to build new skills, take on leadership roles, and connect and grow through mentorship.
• People and Culture: In our dynamic workplace, diversity, equity, and inclusiveness are ingrained in our culture. We're united by a commitment to create an environment where every individual's differences are valued, practices are equitable, fostering a sense of belonging. Our shared values include integrity, respect, teaming, inclusiveness, energy, enthusiasm, courage to lead, and building relationships based on doing the right thing.
• Benefits: Embark on a transformative career journey with us and indulge in a suite of premium benefits, encompassing exclusive health and wellness packages, enticing rewards, and cutting‑edge learning opportunities that empower you to continually grow and excel in your professional and personal development.

Successful candidates advance to a competency‑based interview. If mutual interest persists, a job offer awaits.

“We are an equal opportunity employer and are committed to Diversity, Equity & Inclusion”.

EY | Building a better working world

EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.

Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.

Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.