Bilingual Bot QA Auditor - Conversational AI

PayJoy, Inc; Bogotá; On-site; COP 194,206,000 - 291,309,000; Full time

Benefits offered by this job

100% Health for employees and family

Life Insurance

Indefinite-term contract

20 vacation days, unlimited sick leave

Co-working Travel perk

Professional Development perk

Phone finance, headset, home office

Catered lunches

Job summary

PayJoy, Inc. seeks a Quality Associate BOT to own the quality review of PayJoy's conversational BOT interactions.

This role audits bot-handled conversations end to end, verifies tags and outcomes against actual conversations, documents findings, and tracks optimization releases to ensure fixes land as intended, contributing to a reliable BOT QA program. The role requires 1–2 years in QA or bot auditing, familiarity with Zendesk and Ultimate, and bilingual English/Spanish communication.

Qualifications

• 1–2 years in Quality Assurance, bot/conversational AI auditing, or contact center quality.

Responsibilities

• Complete 2,000 bot conversation reviews a month (≈100 per day).
• Send a weekly findings report covering market results, data gaps, defects, and open tickets.
• Lead biweekly calibration meetings with the optimization team to review findings and priorities.
• Submit improvement tickets in Zendesk with evidence, severity and conversation ID; confirm fixes on ship.
• Monitor optimization releases at 24h and 1 week after go-live to confirm fixes.
• Flag authentication, agent-handoff, and dead-end issues in bot flows for escalation.

Skills

Quality Assurance

Bot auditing

Analytical skills

Tools

Zendesk

Ultimate

CRM/bot platforms

Google Suite

Office Suite

Job description

PayJoy, Inc. seeks a Quality Associate BOT to own the quality review of PayJoy's conversational BOT interactions.

This role audits bot-handled conversations end to end, verifies tags and outcomes against actual conversations, documents findings, and tracks optimization releases to ensure fixes land as intended, contributing to a reliable BOT QA program. The role requires 1–2 years in QA or bot auditing, familiarity with Zendesk and Ultimate, and bilingual English/Spanish communication.