Quality Lead/Supervisor - AI Automation Auditing (BPO/Contact Center)

Sutherland; Mabalacat; In loco; PHP 600.000 - 900.000; Tempo pieno

Descrizione del lavoro

Sutherland is seeking an operational Knowledge AI Quality Lead to oversee QA for AI-driven customer service tools and hybrid support workflows within the partner's contact center operations. You will monitor automated workflows, bot performance, and handovers to ensure accurate, compliant customer experiences.

The role requires hands-on auditing of AI conversations, testing of AI copilot integrations, and delivering weekly improvement insights to the client and product teams.

Competenze

• 2+ years of quality management or auditing in a high-volume BPO/contact center.
• Experience testing, auditing, or evaluating AI chatbots, conversational flows, or automation tools.
• Travel industry experience preferred.
• Familiarity with Lean, Six Sigma, or Root Cause Analysis / 5 Whys.

Mansioni

• Operationalize and execute QA frameworks for AI chatbots, virtual agents, and copilot tools.
• Perform routine pre-release UAT and regression testing across prompts, KB updates, and automated workflow releases.
• Audit live AI-driven conversations, automated resolutions, and handovers to evaluate context retention and accuracy.
• Produce weekly reports and feed improvement insights to client AI leads, KB managers, and engineering.
• Identify KB gaps and propose new intents to broaden containment coverage.

Descrizione del lavoro

Company Description

We are seeking an operational Knowledge AI Quality Lead from our BPO partner to oversee, monitor, and elevate the performance of our AI-driven customer service tools and hybrid support workflows. Positioned within the partner's contact center operations, this role ensures that automated workflows, virtual assistants, agent copilots, and human-assist handovers deliver reliable, accurate, and compliant customer experiences.

The ideal candidate combines operational contact center auditing expertise with a deep understanding of AI conversational dynamics, acting as the frontline bridge between real-time vendor performance data and client-side engineering, product, and knowledge teams.

Job Description

AI QA Framework & Testing

• Operationalize and execute client-approved QA frameworks, evaluation rubrics, and test scripts tailored for AI chatbots, virtual agents, and copilot tools.

• Perform routine pre-release User Acceptance Testing (UAT) and regression testing across prompt deployments, Knowledge Base (KB) updates, and automated workflow releases within the operational environment.

Operational Auditing & Performance Monitoring

• Conduct daily audits of live AI-driven conversations, automated resolutions, and self-service interactions to detect intent misclassifications, system hallucinations, incorrect policy applications, or journey drops.

• Audit the handover bridge between AI and Human-Assist agents to evaluate context retention, CRM data pass-through accuracy, and agent handle times post-transfer.

Closed-Loop Vendor Feedback & Defect Analysis

• Perform root-cause defect analysis on resolution failures, escalation spikes, and recurring customer friction points.

• Synthesize audit data into actionable weekly reports; feed continuous improvement insights to Client AI Automation Leads, KB Managers, and Product Engineering for system retuning and sprint updates.

• Identify out-of-scope traveler queries and KB content gaps, recommending new intents and articles to safely broaden containment coverage.

Operational Metrics & SLA Compliance

• Measure, track, and report on primary operational and AI quality KPIs:

  • Intent Recognition Accuracy
  • Resolution Precision & Answer Accuracy
  • Containment Rate & Defect Rate
  • Handover Context Retention Accuracy
  • Traveler Sentiment Score (TSS) / CSAT Impact

• Hold operational teams and frontline agents accountable for adhering to updated AI copilot workflows and knowledge standards.

Qualifications

Experience

• Contact Center QA Background: 2+ years of experience in quality management, operational auditing, or process improvement within a high-volume BPO/contact center environment.

• AI & Automation Auditing: Demonstrated experience testing, auditing, or evaluating AI chatbots, conversational flows, copilot integrations, or workflow automation tools.

• Travel Industry (Preferred): Prior experience in travel, hospitality, or tech-enabled customer support operations is strongly preferred.

• Methodology: Familiarity with operational process improvement tools (Lean, Six Sigma, Root Cause Analysis / 5 Whys).

Core Competencies & Skills

• Conversational AI Fluency: Deep understanding of natural language processing (NLP/NLU) behavior, intent mapping structures, and failure patterns (e.g., hallucination, drift, misclassification).

• Analytical & Data-Driven: Ability to turn transcript audits, CSS feedback, and performance dashboard data into actionable insights, using AI tools to support analysis and validating outputs for accuracy and relevance.

• Cross-Functional Communication: Clear written and verbal reporting skills to present quality defects and technical feedback to both BPO leadership and client technical teams.

• Quality Standard-Setting: Meticulous attention to detail with an unyielding commitment to brand trust and effortless traveler resolution.