Quality Lead - AI & Automation Auditing (BPO Contact Center)

Sutherland; Angeles; On-site; PHP 800,000 - 1,200,000; Full time

Job summary

Sutherland, a leading BPO contact center partner in Angeles, Philippines, seeks an Operational Knowledge AI Quality Lead to oversee AI-driven tools and hybrid workflows, ensuring accurate and compliant customer experiences.

The role bridges vendor performance data with client engineering, product, and knowledge teams, focusing on QA frameworks, audits, and continuous improvement of conversational AI and copilot workflows.

Qualifications

• 2+ years of QA / quality management in high-volume BPO/contact center environments.
• Experience testing, auditing, or evaluating AI chatbots, conversational flows, copilot integrations, or workflow automation tools.
• Travel industry experience preferred in travel, hospitality, or tech-enabled customer support operations.

Responsibilities

• Oversee AI-driven customer service tools performance within BPO partner operations.
• Conduct daily audits of live conversations, automated resolutions, and self-service interactions to detect issues.
• Perform root-cause analysis on resolution failures and escalation spikes.
• Synthesize audit data into actionable weekly reports for AI/Automation Leads, KB Managers, and Product Engineering.
• Identify KB gaps and new intents to broaden containment coverage.

Skills

QA in contact centers

AI QA

Data analysis

Cross-functional communication

Lean Six Sigma

Job description

Quality Lead - AI & Automation Auditing (BPO Contact Center)

• Full-time

We are seeking an operational Knowledge AI Quality Lead from our BPO partner to oversee, monitor, and elevate the performance of our AI-driven customer service tools and hybrid support workflows. Positioned within the partner's contact center operations, this role ensures that automated workflows, virtual assistants, agent copilots, and human-assist handovers deliver reliable, accurate, and compliant customer experiences.

The ideal candidate combines operational contact center auditing expertise with a deep understanding of AI conversational dynamics, acting as the frontline bridge between real-time vendor performance data and client-side engineering, product, and knowledge teams.

AI QA Framework & Testing

Operationalize and execute client-approved QA frameworks, evaluation rubrics, and test scripts tailored for AI chatbots, virtual agents, and copilot tools.

Perform routine pre-release User Acceptance Testing (UAT) and regression testing across prompt deployments, Knowledge Base (KB) updates, and automated workflow releases within the operational environment.

Operational Auditing & Performance Monitoring

Conduct daily audits of live AI-driven conversations, automated resolutions, and self-service interactions to detect intent misclassifications, system hallucinations, incorrect policy applications, or journey drops.

Audit the handover bridge between AI and Human-Assist agents to evaluate context retention, CRM data pass-through accuracy, and agent handle times post-transfer.

Perform root-cause defect analysis on resolution failures, escalation spikes, and recurring customer friction points.

Synthesize audit data into actionable weekly reports; feed continuous improvement insights to Client AI Automation Leads, KB Managers, and Product Engineering for system retuning and sprint updates.

Identify out-of-scope traveler queries and KB content gaps, recommending new intents and articles to safely broaden containment coverage.

Operational Metrics & SLA Compliance

Measure, track, and report on primary operational and AI quality KPIs:

Intent Recognition Accuracy

Handover Context Retention Accuracy

Traveler Sentiment Score (TSS) / CSAT Impact

Hold operational teams and frontline agents accountable for adhering to updated AI copilot workflows and knowledge standards.

Experience

Contact Center QA Background: 2+ years of experience in quality management, operational auditing, or process improvement within a high-volume BPO/contact center environment.

AI & Automation Auditing: Demonstrated experience testing, auditing, or evaluating AI chatbots, conversational flows, copilot integrations, or workflow automation tools.

Travel Industry (Preferred): Prior experience in travel, hospitality, or tech-enabled customer support operations is strongly preferred.

Methodology: Familiarity with operational process improvement tools (Lean, Six Sigma, Root Cause Analysis / 5 Whys).

Core Competencies & Skills

Conversational AI Fluency: Deep understanding of natural language processing (NLP/NLU) behavior, intent mapping structures, and failure patterns (e.g., hallucination, drift, misclassification).

Analytical & Data-Driven: Ability to turn transcript audits, CSS feedback, and performance dashboard data into actionable insights, using AI tools to support analysis and validating outputs for accuracy and relevance.

Cross-Functional Communication: Clear written and verbal reporting skills to present quality defects and technical feedback to both BPO leadership and client technical teams.

Quality Standard-Setting: Meticulous attention to detail with an unyielding commitment to brand trust and effortless traveler resolution.

All your information will be kept confidential according to EEO guidelines.