Remote Senior QA Engineer – AI Voice, LLM & SIP Testing

IDT Corporation; United States; Remote; USD 120,000 - 160,000; Full time

Benefits offered by this job

Remote work opportunity

Job summary

IDT Corporation is seeking a Senior QA Engineer to own quality for our AI voice virtual agent. You will test non‑deterministic behavior, reason over an LLM, and coordinate SIP-based calls to meet latency targets. The role requires advanced English and hands‑on SIP testing expertise.

You will embed in a Scrum team, drive the end‑to‑end test strategy across LLM, speech, and telephony layers, and extend automated coverage within CI/CD. Remote work is offered with international scope.

Qualifications

• 5+ years in QA experience with complex, integrated products.
• Hands-on testing of SIP/VoIP systems including call flows and signaling.
• Creative test design for non-deterministic systems and live scenarios.
• Proficiency in Python for building automation frameworks.
• Experience testing non-deterministic or probabilistic systems with clear assertions.
• Strong analytical skills to define quality metrics and support release decisions.
• Hands-on with TMS tools and Jira.
• Professional fluency in English with US East hours overlap.

Responsibilities

• Embed within a Scrum team and participate in ceremonies.
• Own end-to-end test strategy for AI voice virtual agent across LLM, speech, and telephony layers.
• Design test scenarios for multi-turn dialogues, interruptions, and escalations.
• Build and maintain an evaluation framework for non-deterministic output.
• Adversarially test the agent for hallucinations and guardrail bypasses.
• Prepare SIP devices and call scenarios to automate end-to-end origination.

Skills

QA experience

Python programming

Playwright

SIP/VoIP testing

Non-deterministic test design

Data-driven decisions

Jira

Tools

JsSIP / WebRTC

SIP softphones

Jenkins

GitHub Actions

Allure TestOps

Grafana

Job description

IDT Corporation is seeking a Senior QA Engineer to own quality for our AI voice virtual agent. You will test non‑deterministic behavior, reason over an LLM, and coordinate SIP-based calls to meet latency targets. The role requires advanced English and hands‑on SIP testing expertise.

You will embed in a Scrum team, drive the end‑to‑end test strategy across LLM, speech, and telephony layers, and extend automated coverage within CI/CD. Remote work is offered with international scope.