Senior SDET - AI Agentic QA for Cloud SaaS

Dialpad, Inc.; Vancouver; Hybride; CAD 151 000 - 175 000; Plein temps

Avantages offerts par ce poste

Competitive salary

Comprehensive benefits

Growth opportunities

Résumé du poste

Dialpad is seeking a Sr. SDET in Agentic QA to own the test automation and quality frameworks supporting our AI Voice Agent services.

You will develop automated tests for end-to-end product experiences, from frontend UI to backend services to APIs to audio/text interactions, ensuring robust automated coverage. You will build substantial automation infrastructure and partner with the development team to make our fast-growing AI platform more testable, stable, and delightful for customers.

Qualifications

• 6+ years of experience in software engineering or SDET roles.
• Experience testing distributed, cloud-native SaaS systems and APIs.
• Hands-on exposure to LLMs or AI/ML systems.

Responsabilités

• Own end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification.
• Design and build automation tooling and frameworks for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations.
• Develop and maintain evaluation frameworks (evals) to measure response quality, accuracy, and hallucination rates.
• Drive automation coverage (80%+ for critical AI workflows) using deterministic + probabilistic validation approaches.
• Integrate AI quality checks into CI/CD pipelines with fast feedback cycles (<15 minutes for PR validation).
• Build tooling for LLM observability and debugging, including prompt tracing and response analysis.
• Partner with Applied AI teams on prompt engineering, model selection and evaluation strategies.
• Design and execute performance and load tests for AI services (latency, throughput, cost efficiency).
• Identify and mitigate risks related to hallucinations, bias, safety and edge cases.
• Define and track AI quality KPIs (task success rates, precision/recall, latency, etc.).
• Participate in design and architecture reviews to ensure systems are testable, observable, and resilient.
• Mentor engineers and contribute to raising the bar on AI quality engineering practices.

Connaissances

SDET experience

Distributed systems testing

AI/ML exposure

CI/CD pipelines

Frontend testing

Formation

Bachelor’s degree in Computer Science or equivalent

Outils

Jenkins

GitHub Actions

LiveKit

Prompt orchestration frameworks

Description du poste

Dialpad is seeking a Sr. SDET in Agentic QA to own the test automation and quality frameworks supporting our AI Voice Agent services.

You will develop automated tests for end-to-end product experiences, from frontend UI to backend services to APIs to audio/text interactions, ensuring robust automated coverage. You will build substantial automation infrastructure and partner with the development team to make our fast-growing AI platform more testable, stable, and delightful for customers.