AI Test Automation Lead

Cognizant; Toronto; Hybrid; CAD 90,000 - 117,000; Full time

Job summary

Cognizant in Canada is seeking an Automation Test Lead to orchestrate AI-powered quality engineering and test automation across insurance and retirement domains. You will shape testing strategies, build scalable frameworks, and integrate testing into CI/CD pipelines to accelerate model updates.

You will mentor engineers, collaborate with cross-functional teams, and advance responsible AI practices while delivering robust automation for UI, API, and data layer tests in a hybrid Toronto/Kitchener

Qualifications

• 10+ years of Quality Engineering experience covering functional, API, integration, and automation testing with a dedicated focus on AI testing and validation.

Responsibilities

• Create and maintain test cases for prompt-driven applications, AI agents, RAG systems, and workflow orchestration platforms
• Design, build, and maintain automated test frameworks for AI applications and services
• Develop automated evaluation pipelines to assess AI responses, workflows, and model behavior
• Drive automation-first quality strategy and lead automation solutions across programs.
• Design scalable automation architectures spanning UI, API, Mobile, and Database layers.
• Develop GenAI-powered solutions for automated test design, code generation, execution, and defect analysis.
• Test AI agents, copilots, APIs, workflow engines, MCP integrations, and tool-calling capabilities
• Leverage AI and ML tools to enhance test coverage, improve efficiency, and reduce regression cycles.
• Leverage GitHub Copilot and Claude to enable AI-assisted automation and enforce CI/CD quality gates.
• Integrate automated AI and regression test suites into CI/CD pipelines to support continuous integration and rapid model updates.
• Collaborate with cross-functional teams and mentor engineers on GenAI, Agentic AI, automation, model deployment, and AI safety best practices

Skills

Test Automation Architecture

GenAI & LLMs

GitHub Copilot

CI/CD & DevOps

Quality Analytics

Stakeholder Management

Strong communication

Tools

Python

Java

AI testing frameworks

MLOps

Claude

Job description

As an Automation Test Lead, you will lead AI-powered Quality Engineering and test automation solutions, including developing testing strategies and frameworks, integrating testing into CI/CD pipelines, validating AI behavior and integrations, and promoting responsible AI practices across complex insurance and retirement systems.

In this role, you will

• Create and maintain test cases for prompt-driven applications, AI agents, RAG systems, and workflow orchestration platforms
• Design, build, and maintain automated test frameworks for AI applications and services
• Develop automated evaluation pipelines to assess AI responses, workflows, and model behavior
• Drive automation-first quality strategy and lead automation solutions across programs.
• Design scalable automation architectures spanning UI, API, Mobile, and Database layers.
• Develop and implement GenAI-powered solutions for automated test design, code generation, execution, and defect analysis.
• Test AI agents, copilots, APIs, workflow engines, MCP integrations, and tool-calling capabilities
• Leverage AI and ML tools to enhance test coverage, improve efficiency, and reduce regression cycles.
• Leverage GitHub Copilot and Claude to enable AI-assisted automation and enforce CI/CD quality gates.
• Integrate automated AI and regression test suites into CI/CD pipelines to support continuous integration and rapid model updates.
• Collaborate with cross-functional teams and mentor engineers on GenAI, Agentic AI, automation, model deployment, and AI safety best practices.

Work model

We believe hybrid work is the way forward as we strive to provide flexibility wherever possible. Based on this role’s business requirements, this is a hybrid position requiring 3 days a week in a client or Cognizant office in Toronto or Kitchener location. Regardless of your working arrangement, we are here to support a healthy work-life balance though our various wellbeing programs.

Note: The working arrangements for this role are accurate as of the date of posting. This may change based on the project you’re engaged in, as well as business and client requirements. Rest assured; we will always be clear about role expectations.

Key Skills: Test Automation Architecture | GenAI & LLMs | GitHub Copilot | CI/CD & DevOps | Quality Analytics | Strong communication, presentation skills | Stakeholder Management

What you need to have to be considered

• 10+ years of Quality Engineering experience covering functional, API, integration, and automation testing with a dedicated focus on AI testing and validation.
• 3+ years in AI/ML systems testing; expert proficiency in AI/ML testing frameworks, MLOps, CI/CD integration, and programming (Python, Java);
• Hands-on GitHub Copilot and Claude experience to enable AI-assisted automation and enforce CI/CD quality gates.
• Hands-on experience with AI platforms and prompt engineering.
• Ability to work independently, have the capability to lead a team and engage with senior business and technology stakeholders.

These will help you stand out

• Exposure to Life Insurance, Retirement domain

Total compensation

We regularly assess market data to ensure we offer a competitive compensation package for our associates. The base salary for this position ranges between CAD 90,000-117,000 per year. Where the successful candidate may fall within the range depends on relevant education, work and/or management experience and other business-related and job-necessary qualifications. This position is also eligible for Cognizant’s discretionary annual performance-based bonus, as well as benefits that support your physical, mental and financial wellbeing.