Lead QA Engineer (AI Voice)

Lingaro Group Philippines; Manila; Presencial; PHP 600.000 - 900.000; Jornada completa

Descripción de la vacante

Lingaro Group Philippines is seeking a senior QA professional to lead testing for a multilingual voice AI platform. You will define the test strategy, drive UAT, and coordinate business testers while evaluating prompts and LLM responses. You will mentor engineers and ensure readiness across markets.

Qualified candidates have extensive QA experience, including voice agents, CI/CD and API testing, plus familiarity with cloud speech services and log analysis. English at least B2 is required.

Formación

• At least 6 years of experience in software QA.
• At least 2 years of experience in testing voice agents or conversational AI in live delivery.
• Has led E2E and UAT, and worked as the lone or lead QA with limited guidance.
• Experience in international or distributed teams.
• Voice AI testing: speech recognition and speech output, latency, interruptions, accents, noise.
• Voice quality metrics such as word error rate, intent accuracy and task completion.
• Practical understanding of how LLMs work and fail. Deep LLM engineering is not needed.
• Test automation in Python or JavaScript/TypeScript, with CI/CD and API testing (e.g. Postman).
• Use of cloud speech services (Azure, Google, AWS or similar) to generate test calls.
• Practical experience with Jira or Azure DevOps, and a test management tool.
• Practical experience with troubleshooting from logs, transcripts and call recordings.
• Voice testing platforms (Hamming, Cekura, Coval, Botium) is a plus.
• Experience with LLM evaluation tools (Promptfoo, DeepEval, LangSmith) is a plus.
• Experience with Telephony basics (SIP, Twilio, Azure Communication Services) is a plus. ISTQB certification is a plus.
• Experience in Responsible-AI testing (PII, bias, safety) is a plus.
• Experience validating customer journeys across multiple markets is a plus.
• Structured, recommendation-first communication with technical and business stakeholders.
• English at least B2.

Responsabilidades

• Design the voice AI test strategy and process, from speech input through speech recognition (ASR), the LLM, tools and integrations, to speech output.
• Define the automation approach and how it scales, including multilingual coverage of around 100 languages.
• Test both predictable behaviour (intents, routing, integrations) and non-deterministic AI behaviour.
• Evaluate prompts and LLM responses, using test datasets and scoring criteria.
• Lead UAT: planning, scenarios, test data, coordination of business testers and sign-off recommendation.
• Gather business acceptance criteria and turn them into measurable targets.
• Own defect and risk management, and give release go/no-go recommendations.
• Replan and communicate options when development is delayed.
• Challenge stakeholders on readiness and quality, and mentor the Regular engineer.

Conocimientos

Software QA

Voice AI testing

UAT leadership

CI/CD

Python

JavaScript/TypeScript

API testing

Cloud speech services

Jira/Azure DevOps

Log analysis

Educación

ISTQB certification

Herramientas

Hamming

Cekura

Coval

Botium

Promptfoo

DeepEval

LangSmith

SIP

Twilio

Azure Communication Services

Descripción del empleo

Growth through diversity, equity, and inclusion. As an ethical business, we do what is right — including ensuring equal opportunities and fostering a safe, respectful workplace for each of us. We believe diversity fuels both personal and business growth. We're committed to building an inclusive community where all our people thrive regardless of their backgrounds, identities, or other personal characteristics.

Tasks:

• Design the voice AI test strategy and process, from speech input through speech recognition (ASR), the LLM, tools and integrations, to speech output.

• Define the automation approach and how it scales, including multilingual coverage of around 100 languages.

• Test both predictable behaviour (intents, routing, integrations) and non-deterministic AI behaviour.

• Evaluate prompts and LLM responses, using test datasets and scoring criteria.

• Lead UAT: planning, scenarios, test data, coordination of business testers and sign-off recommendation.

• Gather business acceptance criteria and turn them into measurable targets.

• Own defect and risk management, and give release go/no-go recommendations.

• Replan and communicate options when development is delayed.

• Challenge stakeholders on readiness and quality, and mentor the Regular engineer.

What We're Looking For:

• At least 6 years of experience in software QA.

• At least 2 years of experience in testing voice agents or conversational AI in live delivery.

• Has led E2E and UAT, and worked as the lone or lead QA with limited guidance.

• Experience in international or distributed teams.

• Voice AI testing: speech recognition and speech output, latency, interruptions, accents, noise.

• Voice quality metrics such as word error rate, intent accuracy and task completion.

• Practical understanding of how LLMs work and fail. Deep LLM engineering is not needed.

• Test automation in Python or JavaScript/TypeScript, with CI/CD and API testing (e.g. Postman).

• Use of cloud speech services (Azure, Google, AWS or similar) to generate test calls.

• Practical experience with Jira or Azure DevOps, and a test management tool.

• Practical experience with troubleshooting from logs, transcripts and call recordings.

• Voice testing platforms (Hamming, Cekura, Coval, Botium) is a plus.

• Experience with LLM evaluation tools (Promptfoo, DeepEval, LangSmith) is a plus.

• Experience with Telephony basics (SIP, Twilio, Azure Communication Services) is a plus. ISTQB certification is a plus.

• Experience in Responsible-AI testing (PII, bias, safety) is a plus.

• Experience validating customer journeys across multiple markets is a plus.

• Structured, recommendation-first communication with technical and business stakeholders.

• English at least B2.

Missing one or two of these qualifications? We still want to hear from you! If you bring a positive mindset, we'll provide an environment where you feel valued and empowered to learn and grow.