Lead AI Test Automation Architect

ITMC Systems, Inc; Toronto; On-site; CAD 150,000 - 190,000; Full time

Job summary

ITMC Systems, Inc. seeks a Lead AI Test Automation Architect to drive the design, development, and optimization of AI-powered quality engineering solutions.

You will lead advanced automation frameworks, mentor teams, and advance MCP-driven, context-aware automation across the SDLC in a cloud-enabled environment. The role requires deep hands-on experience with LLMs, LangChain, PyTest, Playwright, and ML tooling, and 10–14 years in automation and AI QE.

Qualifications

• Hands-on AI and LLM experience with prompt engineering, RAG architectures, and model evaluation.
• Proficiency with LangChain, Hugging Face, Transformers, and OpenAI/Ollama APIs.
• Experience with agentic frameworks such as LangGraph, AutoGen, or CrewAI.
• Design GenAI-powered QE platforms, AI agents, and autonomous testing workflows.
• Implement MCP-driven, context-aware automation and intelligent CI/CD decision engines.
• Strong coding skills in Python, TypeScript, or Java.

Responsibilities

• Hands-on AI expertise in LLMs, prompt engineering, RAG architectures, and model evaluation techniques.
• Architect and enhance GenAI-powered QE solutions, AI agents, and autonomous testing workflows.
• Design scalable automation frameworks and CI/CD decision engines across QE automation, AI agents, RAG pipelines, and MCP-enabled workflows.
• Build data/embedding pipelines and optimize retrieval for RAG architectures.
• Lead and mentor teams; drive AI adoption and quality engineering best practices.
• Collaborate with developers, SMEs, and product management to align QE solutions with business goals.

Skills

AI Expertise

AI Tooling

Agentic Frameworks

GenAI Solutions

Context-Aware Automation

Programming Skills

Automation Frameworks

UI Automation

API Testing

Performance Testing

AI Asset Generation

ML Libraries

RAG Pipelines

MLOps CI/CD

Tools

LangChain

Hugging Face

Transformers

OpenAI APIs

Ollama

LangGraph

AutoGen

CrewAI

Playwright

Selenium

PyTest

Requests

RestAssured

PyTorch

TensorFlow

Scikit-Learn

NLTK

OpenCV

Job description

Job Title: Lead AI Test Automation Architect

Experience: 10–14 Years

Role Type: Architect / Senior SDET / AI QE Technical Lead

Role Overview

ITMC Systems is seeking an experienced Lead AI Test Automation Architect to drive the design, development, and optimization of next-generation, AI-powered quality engineering (QE) solutions and platforms.

This role combines deep technical expertise in test automation engineering with hands‑on mastery of Large Language Models (LLMs), agentic AI frameworks, and enterprise‑grade AI tooling. In this role, you will define overall quality engineering strategies, design scalable automation frameworks, mentor technical teams, and pioneer innovation across QE automation, AI agents, RAG pipelines, and MCP‑enabled intelligent workflows.

Key Responsibilities & Required Skills

1. AI, LLMs & Agentic Systems

• Hands‑on AI Expertise: Strong, direct experience working with LLMs, prompt engineering, RAG architectures, vector databases, and model evaluation techniques.
• AI Tooling & Frameworks: Proficiency with LangChain, Hugging Face, Transformers, and OpenAI/Ollama APIs.
• Agentic Frameworks: Practical experience with agentic frameworks such as LangGraph, AutoGen, or CrewAI.
• GenAI QE Solutions: Architect and enhance GenAI‑powered quality engineering platforms, AI agents, and autonomous testing workflows.
• Context‑Aware Automation: Implement Model Context Protocol (MCP)‑driven, context‑aware automation and intelligent CI/CD decision engines.

2. Automation Engineering & Test Architecture

• Core Development: Strong coding skills in Python, TypeScript, or Java.
• Framework Design: Architect and maintain enterprise‑level automation frameworks across:
• UI Automation: Playwright, Selenium
• API Testing: PyTest, Requests, RestAssured
• Performance Testing: JMeter, Locust
• AI Asset Generation: Develop prompt‑optimized, AI‑generated test assets, test cases, and modern validation mechanisms.
• Machine Learning Libraries: Experience with PyTorch, TensorFlow, Scikit‑Learn, and NLP/CV libraries (e.g., NLTK, BART, OpenCV).
• Data & RAG Pipelines: Build robust data/embedding pipelines and optimize retrieval mechanisms for RAG architectures.
• MLOps Integration: Implement end‑to‑end CI/CD for ML models, including model versioning, evaluation metrics, and retraining workflows.

4. Cloud, DevOps & Governance

• Cloud Platforms: Deep understanding of AWS, Azure, or GCP cloud architectures and their respective AI/ML services.
• CI/CD Integration: Seamlessly integrate test automation pipelines using GitHub Actions, Azure DevOps, or Jenkins.
• Security & Governance: Ensure all AI/automation infrastructure and environments are scalable, secure, and compliant with enterprise governance standards.

5. Leadership & Delivery Excellence

• Technical Mentorship: Provide technical leadership and mentor cross‑functional teams on AI adoption and automation best practices.
• Stakeholder Alignment: Collaborate closely with developers, SMEs, and product management to align quality engineering solutions with business goals.
• Quality Strategy: Drive feature prioritization, end‑to‑end quality strategy, defect triage, and solution architecture.
• Full SDLC Engagement: Work across the entire Software Development Life Cycle (SDLC) in an Agile/Scrum environment, actively contributing to test design, execution, and analysis.