Product Quality Analyst | Remote

Crossing Hurdles; United States; Remote; CAD 70,000 - 90,000; Full time

Job summary

A leading technology company in Canada is seeking an experienced UX evaluator to design evaluation rubrics for AI model outputs and assess user experience. The candidate will execute product validation tasks, analyze AI outputs for usability issues, and collaborate across teams to develop actionable recommendations. Strong communication and analytical skills are essential, along with experience in evaluation frameworks. This role is fully remote, ideal for a self-directed professional.

Qualifications

• Exceptional written and verbal communication skills.
• Strong analytical skills focused on user risk and usability.
• Experience in developing evaluation frameworks focusing on UX and safety.
• Ability to document findings in a user-friendly manner.
• Comfortable collaborating in a fast-paced environment.
• Self-directed and reliable in a fully remote setting.

Responsibilities

• Design and develop evaluation rubrics for AI model outputs.
• Execute product validation tasks to identify UX risks.
• Analyze AI outputs to flag usability issues.
• Collaborate with teams to define product validation standards.
• Translate user insights into actionable recommendations.
• Document results clearly for stakeholder understanding.
• Monitor AI product usage to surface emerging UX risks.

Skills

Exceptional communication skills

Strong analytical skills

Experience with evaluation frameworks

Ability to document findings clearly

Collaboration in fast-paced environments

Self-directed work style

Job description

• Design and develop robust evaluation rubrics to assess AI model outputs from a user experience (UX) perspective.
• Execute product validation tasks to identify risks that could negatively impact user experience, safety, or accuracy.
• Analyze AI outputs to flag usability issues, inconsistencies, or potential failure modes.
• Collaborate with cross-functional teams to define, refine, and uphold product validation and safety standards.
• Translate user insights and evaluation findings into clear, actionable recommendations for model improvement.
• Document results and recommendations with clarity to ensure stakeholders understand user impact.
• Continuously monitor evolving AI product usage and proactively surface emerging UX risks or concerns.

Requirements

• Exceptional written and verbal communication skills with a strong focus on clarity and precision.
• Strong analytical skills and attention to detail when evaluating user risk and usability issues.
• Experience developing and applying evaluation frameworks balancing UX, safety, and performance.
• Ability to clearly document complex findings in an actionable, user-friendly manner.
• Comfortable collaborating across teams in a fast-paced, evolving project environment.
• Self-directed and reliable in a fully remote setting.