Remote: Senior QA Automation Engineer, SIP/VoIP & AI Voice

IDT; United States; Remote; USD 110,000 - 165,000; Full time

Benefits offered by this job

Remote work opportunity

Competitive salary

Career growth

Training & conferences

Referral program

English classes

Hardware provided

Job summary

IDT Technologies is seeking an experienced Senior QA Engineer to own quality for an AI voice virtual agent. You will design tests for non-deterministic behavior, include SIP/VoIP testing, and drive evaluation across LLM, speech, and telephony layers.

Remote work from the US is offered with a strong emphasis on automation and CI/CD integration. You will embed within a Scrum team, own end-to-end test strategy, and develop frameworks to measure metrics like task completion, transcription accuracy,

Qualifications

• 5+ years in QA and hands-on ownership of a complex product.
• Experience testing SIP/VoIP systems and call flows.
• Creativity in test design for live scenarios.
• Proficiency in Python to build automation frameworks.
• Experience testing non-deterministic systems with robust assertions.
• Strong analytical skills to define and justify quality metrics.
• Hands-on with TMS tools (Allure/JA) and Jira.
• Professional fluency in English, with US Eastern overlap.

Responsibilities

• Embed fully within a Scrum team and own end-to-end test strategy for AI voice agent.
• Design test scenarios for multi-turn conversations, interruptions, and escalation.
• Build an evaluation framework for non-deterministic output and regression detection.
• Adversarially test the agent for hallucinations, grounding, and prompt injections.
• Prepare SIP devices and call scenarios to automate end-to-end origination.
• Diagnose failures across LLM, ASR, TTS, media path, and routing layers.
• Extend automated coverage and integrate tests into CI/CD as release gates.
• Define core quality metrics: task completion, accuracy, latency, escalation.

Skills

QA

Python

SIP/VoIP testing

Test design

Analytics

English fluency

Tools

pytest

Playwright

JsSIP/WebRTC

SIP softphones

Allure TestOps

Jira

Jenkins & GitHub Actions

Grafana

LLM evaluation tooling

Job description

IDT Technologies is seeking an experienced Senior QA Engineer to own quality for an AI voice virtual agent. You will design tests for non-deterministic behavior, include SIP/VoIP testing, and drive evaluation across LLM, speech, and telephony layers.

Remote work from the US is offered with a strong emphasis on automation and CI/CD integration. You will embed within a Scrum team, own end-to-end test strategy, and develop frameworks to measure metrics like task completion, transcription accuracy,