Senior QA Automation Engineer — AI Voice & SIP Testing

IDT; Bogotá; Remote; COP 280,260,000 - 435,961,000; Full time

Benefits offered by this job

Remote work opportunity

B2B Employment (gross)

Competitive salary with annual review

Hardware provided

Career growth & training assistance

Referral program

Job summary

IDT Technologies is seeking an experienced Senior QA Engineer to own quality for our AI voice virtual agent. You will design end-to-end test strategies across LLM, speech, and telephony layers, build evaluation frameworks for non-deterministic outputs, and drive automation through CI/CD.

This role requires testing live SIP calls, handling diverse audio conditions, and delivering data-driven quality insights.

Qualifications

• 5+ years in QA ownership of a complex product.
• Experience testing SIP/VoIP systems and tracing.
• Creative test design for live scenarios.
• Proficiency in Python to build automation.
• Experience testing non-deterministic systems with assertions.

Responsibilities

• Embed within a Scrum team and own end-to-end test strategy.
• Design test scenarios for multi-turn dialogues and barge-in.
• Build evaluation framework for non-deterministic output.
• Adversarially test against hallucinations and data leakage.
• Prepare SIP devices and call scenarios for automation.
• Diagnose failures across LLM, ASR, TTS, media, and routing.
• Extend automated coverage and CI/CD integration.
• Define core quality metrics and track in dashboards.
• Maintain test coverage in the TMS with traceability.
• Communicate quality risks with data-backed evidence.

Skills

QA experience

SIP/VoIP testing

Python programming

Non-deterministic testing

English fluency

Tools

Python with pytest

Playwright

JsSIP/WebRTC

SIP softphones/devices

Allure TestOps

Jira

Jenkins & GitHub Actions

Grafana

Job description

IDT Technologies is seeking an experienced Senior QA Engineer to own quality for our AI voice virtual agent. You will design end-to-end test strategies across LLM, speech, and telephony layers, build evaluation frameworks for non-deterministic outputs, and drive automation through CI/CD.

This role requires testing live SIP calls, handling diverse audio conditions, and delivering data-driven quality insights.