Test Manager AI Architect / Manager

PwC; Bengaluru; On-site; INR 260,000 - 380,000; Full time

Job summary

PwC Bengaluru is seeking a seasoned AI QA Architect to lead the design and delivery of AI-driven QA platforms across enterprise programs. You will build AI bots for testing, analytics, and governance, shaping the AI strategy for the ETMS practice while ensuring scalability and observability.

You will lead teams, integrate AI into CI/CD, and translate AI capabilities into tangible business value for clients.

Qualifications

• Bachelors or master’s degree in computer science, Engineering, Data Science, or equivalent experience.
• 6+ years of experience in QA automation or software engineering.
• 3+ years of experience working with AI-driven automation or intelligent testing frameworks.
• Minimum 2 years of hands-on experience in building or testing AI-based tools or platforms.

Responsibilities

• Architect and own AI-driven QA platforms across enterprise programs.
• Build, productize, and deploy AI Bots / AI Agents for testing, analytics, and quality decisioning.
• Define and drive the end-to-end AI strategy and roadmap for the ETMS practice.
• Provide hands-on technical leadership in AI agents, GenAI architectures, and intelligent automation frameworks.
• Design enterprisewide LLM integration strategies across test automation frameworks, internal QA tools, and managed QA platforms.
• Establish AI-driven quality metrics and evaluation frameworks using ML metrics (precision, recall, F1, confusion matrix).
• Lead evaluation, selection, onboarding, and governance of AI tools and platforms.
• Integrate AI solutions into CI/CD pipelines for predictive quality, analytics, and automation.
• Drive stakeholder communication and AI narrative building, translating complex AI capabilities into business value.
• Define data strategy and governance models for AI-enabled QA solutions.
• Mentor and grow AI architects, test managers, and technical leaders within the practice.

Skills

QA automation experience

AI-driven automation

AI-based tools testing

Education

Bachelor's or Master's in CS/Engineering/Data Science

Tools

LangChain + LangSmith / LlamaIndex

DeepEval / RAGAS

MLflow / Comet ML

Scikit-learn metrics

Azure OpenAI / AWS Bedrock

AutoGen

Job description

Key Responsibilities:

• Architect and own AIdriven QA platforms across enterprise programs, ensuring scalability, observability, and governance
• Build, productize, and deploy AI Bots / AI Agents for testing, analytics, and quality decisioning
• Define and drive the endtoend AI strategy and roadmap for the ETMS practice
• Provide handson technical leadership in AI agents, GenAI architectures, and intelligent automation frameworks
• Design enterprisewide LLM integration strategies across:
• Test automation frameworks
• Internal QA tools
• Managed QA platforms
• enabling intelligent automation, AI observability, and quality governance
• Establish AIdriven quality metrics and evaluation frameworks using ML metrics (precision, recall, F1, confusion matrix)
• Lead evaluation, selection, onboarding, and governance of AI tools and platforms
• Integrate AI solutions into CI/CD pipelines for predictive quality, analytics, and automation
• Drive stakeholder communication and AI narrative building, translating complex AI capabilities into business value
• Define data strategy and governance models for AIenabled QA solutions
• Mentor and grow AI architects, test managers, and technical leaders within the practice

Qualifications and Skills:

• Bachelors or master’s degree in computer science, Engineering, Data Science, or equivalent experience.
• 6+ years of experience in QA automation or software engineering.
• 3+ years of experience working with AIdriven automation or intelligent testing frameworks.
• Minimum 2 years of handson experience in building or testing AIbased tools or platforms.

Tools & Technologies

Must Have Tools

• LangChain + LangSmith / LlamaIndex
• DeepEval / RAGAS
• MLflow / Comet ML
• Scikitlearn metrics
• Azure OpenAI / AWS Bedrock
• AutoGen

Good To Have Tools

• Advanced AutoGen setups
• Predictive analytics tools
• AI observability platforms
• Data governance platforms
• CI/CD AI integration tools
• LucidChart / Miro

Nice To Have Tools

• Custom LLM finetuning
• Opensource LLM hosting
• Domainspecific AI stacks
• Multicloud AI tooling
• Researchgrade ML libraries