Remote Senior QA Automation Engineer - AI Voice Testing

IDT; United States; Remote; USD 120,000 - 160,000; Full time

Benefits offered by this job

Remote work

Annual performance review

Training & conference funding

hardware support

English classes

Referral incentives

Job summary

IDT is seeking a QA lead to own the quality strategy for an AI-powered voice product. You will design end-to-end test coverage across language models, speech services, and telephony, prioritizing automation and CI/CD integration.

You will assess variability in model outputs, build robust assertions, and monitor metrics such as accuracy, latency, transcription quality, and escalation correctness. Remote work options available for US-time overlap.

Qualifications

• 5+ years of QA ownership for a complex, integrated production product.
• Hands-on SIP/VoIP testing experience or related signaling knowledge.
• Strong Python skills to build and maintain automation with pytest.

Responsibilities

• Own end-to-end QA strategy for an AI-powered voice product.
• Lead test design for multi-turn conversations, interruptions, escalation, and recovery.
• Develop evaluation approaches for variable model outputs and regression detection.
• Test for hallucinations, prompt injection, data exposure, and guardrails.

Skills

QA leadership

Python automation

Test strategy design

Analytical thinking

Agile/Scrum

Tools

pytest

Jira

CI/CD pipelines

Job description

IDT is seeking a QA lead to own the quality strategy for an AI-powered voice product. You will design end-to-end test coverage across language models, speech services, and telephony, prioritizing automation and CI/CD integration.

You will assess variability in model outputs, build robust assertions, and monitor metrics such as accuracy, latency, transcription quality, and escalation correctness. Remote work options available for US-time overlap.