Principal AI Quality Developer

Jobgether; Canada; Hybrid; CAD 80,000 - 100,000; Full time

Job summary

A technology firm is seeking a Principal AI Quality Developer in Canada to design and implement testing strategies for AI and ML systems. This role will focus on establishing quality initiatives and ensuring AI capabilities meet industry standards. The ideal candidate will have significant experience in quality engineering, strong programming skills in Python, and be adept at collaborating in a dynamic environment. This position offers the chance to work on high-impact AI technologies and flexible working opportunities.

Benefits

Flexible working hours

Remote work opportunities

Collaborative and inclusive culture

Continuous learning opportunities

Access to advanced tools for AI testing

Qualifications

• 7+ years of experience in quality engineering or test automation.
• Strong programming skills in Python.
• Hands-on experience in testing AI or ML systems.

Responsibilities

• Define and drive the quality strategy for applied AI features.
• Conduct test planning and acceptance criteria for AI capabilities.
• Build automated evaluation frameworks for LLM and ML outputs.

Skills

Quality engineering

Test automation

Python programming

Collaboration

Debugging complex systems

AI testing

Education

Bachelor's degree in Computer Science or related field

Job description

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Principal AI Quality Developer. In this role, you will design and implement testing and evaluation strategies for applied AI and ML systems across various products. Your work will directly affect the quality, reliability, and performance of AI features. By partnering closely with engineering teams, you will help establish a robust framework for evaluating AI capabilities, ensuring they meet customer expectations and industry standards. This is a high-impact position that offers the opportunity to lead quality initiatives in a dynamic environment.

Accountabilities

• Define and drive the quality strategy for applied AI features.
• Conduct test planning and acceptance criteria for AI capabilities.
• Build automated evaluation frameworks for LLM and ML outputs.
• Establish evaluation metrics and scoring for AI behavior.
• Validate workflows and behaviors across various edge cases.
• Lead safety testing in collaboration with security teams.
• Implement post-deployment validation and monitoring.
• Perform issue triage for AI-related defects.
• Mentor engineering teams on AI testing practices.
• Collaborate to embed responsible AI practices in workflows.

Requirements

• 7+ years of experience in quality engineering or test automation.
• Strong programming skills in Python.
• Hands-on experience in testing AI or ML systems.
• Practical knowledge of LLM‑based system validation.
• Experience defining acceptance criteria for engineering projects.
• Proven debugging skills across complex systems.
• Strong communication skills in a collaborative environment.
• Bachelor's degree in Computer Science or related field.

Benefits

• Flexible working hours and remote work opportunities.
• Opportunity to work on high-impact AI technologies.
• Collaborative and inclusive company culture.
• Continuous learning and professional development opportunities.
• Possibility to influence product development processes.
• Access to advanced tools and technologies for AI testing.

Why Apply Through Jobgether?

We use an AI‑powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top‑fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre‑contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.