Lead AI Voice QA Engineer — Remote, Growth & Learning

Lingaro

Polska

Hybrid

PLN 180,000 - 240,000

Full time

5 days ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Lingaro seeks an experienced QA lead to design and oversee voice AI testing across multilingual environments and integrations. You will drive end-to-end quality, implement scalable automation, and lead UAT with cross-functional teams.

The role emphasizes collaboration with stakeholders and mentoring engineers. Applicants should bring 6+ years in software QA, plus hands-on testing of voice agents, CI/CD and API testing experience, and strong English communication.

Qualifications

6+ years of experience in software QA.
2+ years testing voice agents or conversational AI in live delivery.
Led E2E and UAT, and worked as the lone or lead QA with limited guidance.
Experience in international or distributed teams.
Voice AI testing: speech recognition and speech output, latency, interruptions, accents, noise.
Voice quality metrics such as word error rate, intent accuracy and task completion.
Practical understanding of how LLMs work and fail. Deep LLM engineering is not needed.
Test automation in Python or JavaScript/TypeScript, with CI/CD and API testing (e.g. Postman).
Use of cloud speech services (Azure, Google, AWS) to generate test calls.
Practical experience with Jira or Azure DevOps, and a test management tool.
Practical experience with troubleshooting from logs, transcripts and call recordings.
Voice testing platforms (Hamming, Cekura, Coval, Botium) is a plus.
Experience with LLM evaluation tools (Promptfoo, DeepEval, LangSmith) is a plus.
Experience with Telephony basics (SIP, Twilio, Azure Communication Services) is a plus.
ISTQB certification is a plus.
Experience in Responsible-AI testing (PII, bias, safety) is a plus.
Experience validating customer journeys across multiple markets is a plus.
Structured, recommendation-first communication with technical and business stakeholders.
English at least B2.

Responsibilities

Design the voice AI test strategy and process from input to speech output.
Define the automation approach and multilingual coverage for ~100 languages.
Test both deterministic intents and non-deterministic AI behaviour.
Evaluate prompts and LLM responses using datasets and scoring criteria.
Lead UAT: planning, scenarios, test data, coordination and sign-off.
Translate business acceptance criteria into measurable targets.
Own defect and risk management, and provide go/no-go release recommendations.
Replan and communicate options when development delays arise.
Challenge stakeholders on readiness and quality, and mentor the Regular engineer.