Remote Senior QA for AI Voice & SIP Automation

IDT

New Jersey

Remote

MXN 700,000 - 1,100,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Remote work opportunity!
B2B Employment ($, gross)
Competitive salary with annual review
Company-sponsored English classes

Job summary

IDT Technologies in Mexico is seeking a Senior QA Engineer to own the quality of an AI voice virtual agent, combining LLM reasoning with SIP/VoIP testing across the full stack. The role emphasizes designing test scenarios for complex conversations and automating evaluation of non-deterministic outputs.

You will drive end-to-end testing, from data-driven evaluation to latency and escalation metrics, while ensuring robust telemetry and release-grade quality in a fast-paced telephony environment.

Qualifications

  • 5+ years in QA
  • Practical experience testing SIP/VoIP systems
  • Demonstrated creativity in test design
  • Proficiency in Python for automation
  • Experience testing non-deterministic or probabilistic systems
  • Strong analytical skills with data-driven decision making
  • Hands-on experience with Jira and TMS tools
  • Fluent English (spoken and written) with overlap to US Eastern hours

Responsibilities

  • Embed fully within a Scrum team and own the end-to-end test strategy for the AI voice virtual agent across LLM, speech, and telephony layers.
  • Design test scenarios for complex conversational behavior including multi-turn dialogues, interruptions, and escalation paths.
  • Build and maintain an evaluation framework for non-deterministic output and regression detection for prompts, models, or retrieval changes.
  • Adversarially test the agent against hallucinations, grounding issues, prompt injections and data disclosure risks.
  • Prepare and provision SIP devices and call scenarios to automate call origination end-to-end.
  • Diagnose failures across LLM, ASR, TTS, media path, and call routing layers.
  • Extend automated coverage and integrate test suites into CI/CD as release gates.
  • Define and track core quality metrics and maintain test coverage traceability.

Job description

IDT Technologies in Mexico is seeking a Senior QA Engineer to own the quality of an AI voice virtual agent, combining LLM reasoning with SIP/VoIP testing across the full stack. The role emphasizes designing test scenarios for complex conversations and automating evaluation of non-deterministic outputs.

You will drive end-to-end testing, from data-driven evaluation to latency and escalation metrics, while ensuring robust telemetry and release-grade quality in a fast-paced telephony environment.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote: Senior QA Automation Engineer, SIP/VoIP & AI Voice
Remote: Senior QA Automation Engineer, SIP/VoIP & AI Voice

IDT • United States

Remote
USD 110,000 - 165,000
Remote work opportunity
Competitive salary
Career growth
+4
Remote Senior QA Engineer – AI Voice, LLM & SIP Testing
Remote Senior QA Engineer – AI Voice, LLM & SIP Testing

IDT Corporation • United States

Remote
USD 120,000 - 160,000
Remote work opportunity
Remote Senior QA Automation Engineer - AI Voice Testing
Remote Senior QA Automation Engineer - AI Voice Testing

IDT • United States

Remote
USD 120,000 - 160,000
Remote work
Annual performance review
Training & conference funding
+3
Senior QA Automation Engineer
Senior QA Automation Engineer

IDT • United States

Remote
USD 120,000 - 160,000
Remote work
Annual performance review
Training & conference funding
+3
Senior QA Automation Engineer
Senior QA Automation Engineer

IDT • New Jersey

Remote
MXN 700,000 - 1,100,000
Remote work opportunity!
B2B Employment ($, gross)
Competitive salary with annual review
+1
Senior QA Automation Engineer
Senior QA Automation Engineer

IDT Corporation • United States

Remote
USD 120,000 - 160,000
Remote work opportunity
Senior QA Automation Engineer - SIP/VOIP
Senior QA Automation Engineer - SIP/VOIP

IDT • United States

Remote
USD 110,000 - 165,000
Remote work opportunity
Competitive salary
Career growth
+4
Senior QA Engineer, Voice & LLM for Healthcare AI
Senior QA Engineer, Voice & LLM for Healthcare AI

Vi • United States

Remote
USD 120,000 - 180,000
Senior QA Engineer - Automation & AI, Remote
Senior QA Engineer - Automation & AI, Remote

TTEC Digital • United States

Remote
USD 120,000 - 180,000
Remote QA Engineer: Conversational AI & Voice Testing
Remote QA Engineer: Conversational AI & Voice Testing

Miratech1 • United States

Remote
USD 90,000 - 130,000
Remote work
Health insurance
Relocation program
+2