AI Product Lead: Agents, Evaluation & Audit-Ready Platform

Jobtailor; Toronto; On-site; CAD 120,000 - 180,000; Full time

Job summary

Jobtailor is seeking a senior AI-focused product leader to own the applied-science delivery roadmap and coordinate cross-team workstreams for the Caseware AI platform. You will set evaluation baselines, partner with audit professionals, and translate problem statements into concrete requirements and acceptance criteria.

The role requires hands-on expertise in LLMs, RAG, embeddings, and agentic patterns, with a strong bias for high-accuracy AI outputs in regulated contexts.

Qualifications

• 6+ years of experience in technology-focused product management or product development.
• 3–4 years working deeply with AI or machine learning–driven products.
• Proven ability to drive delivery across multiple technical teams and cross-team coordination.
• Experience defining acceptance bars, running human-in-the-loop validation, and using results to decide what ships.

Responsibilities

• Own the applied-science delivery roadmap and cross-team orchestration for the AI platform.
• Define and own product health metrics and evaluation baselines.
• Shadow audit workflows, validate prototypes, and identify actionable failure modes.
• Collaborate with domain experts, engineers, and external partner firms to ensure quality outcomes.
• Make autonomous product decisions and own product outcomes.

Skills

Product management

AI product development

Cross-team collaboration

Delivery leadership

Technical fluency in LLMs

Tools

LLMs

RAG

Embeddings

Job description

Jobtailor is seeking a senior AI-focused product leader to own the applied-science delivery roadmap and coordinate cross-team workstreams for the Caseware AI platform. You will set evaluation baselines, partner with audit professionals, and translate problem statements into concrete requirements and acceptance criteria.

The role requires hands-on expertise in LLMs, RAG, embeddings, and agentic patterns, with a strong bias for high-accuracy AI outputs in regulated contexts.