Remote Senior QA Engineer: AI Voice & Telephony Testing

IDT Corporation

New Jersey

On-site

MXN 700,000 - 1,000,000

Full time

9 days ago

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Benefits offered by this job

Remote work opportunity

Job summary

IDT Corporation in Mexico is seeking an experienced Senior QA Engineer to own quality for our AI voice virtual agent, spanning LLM, speech and telephony layers. You will design evaluation strategies for non-deterministic outputs and ensure latency constraints are met.

This role requires hands-on SIP/VoIP testing, Python automation, Playwright/WebRTC, and collaboration within a Scrum team. Remote work available with overlap with US eastern hours.

Qualifications

5+ years in QA
Proficiency in Python
Experience testing SIP/VoIP systems
Experience testing non-deterministic systems
Professional fluency in English

Responsibilities

Embed within a Scrum team and own end-to-end test strategy
Design test scenarios for complex conversational behavior
Build evaluation framework for non-deterministic output
Adversarially test the agent
Prepare and provision SIP devices and call scenarios
Diagnose failures across the stack (LLM, ASR, TTS, media, routing)
Extend automated coverage and CI/CD integration
Define and track core quality metrics
Maintain test coverage in the TMS with traceability
Communicate quality risks to stakeholders

Skills

Python

English fluency

SIP/VoIP testing

Test design

Non-deterministic systems

Jira

CI/CD

LLM testing

Tools

pytest

Playwright

JsSIP/WebRTC

SIP softphones

Allure TestOps

Jenkins

GitHub Actions

Grafana

Job description

This role requires hands-on SIP/VoIP testing, Python automation, Playwright/WebRTC, and collaboration within a Scrum team. Remote work available with overlap with US eastern hours.

Get your free, confidential resume review.

or drag and drop your file here.