Technical Product Manager – AI Products

Jobtailor; Toronto; On-site; CAD 120,000 - 180,000; Full time

Job summary

Jobtailor is seeking a senior AI-focused product leader to own the applied-science delivery roadmap and coordinate cross-team workstreams for the Caseware AI platform. You will set evaluation baselines, partner with audit professionals, and translate problem statements into concrete requirements and acceptance criteria.

The role requires hands-on expertise in LLMs, RAG, embeddings, and agentic patterns, with a strong bias for high-accuracy AI outputs in regulated contexts.

Qualifications

• 6+ years of experience in technology-focused product management or product development.
• 3–4 years working deeply with AI or machine learning–driven products.
• Proven ability to drive delivery across multiple technical teams and cross-team coordination.
• Experience defining acceptance bars, running human-in-the-loop validation, and using results to decide what ships.

Responsibilities

• Own the applied-science delivery roadmap and cross-team orchestration for the AI platform.
• Define and own product health metrics and evaluation baselines.
• Shadow audit workflows, validate prototypes, and identify actionable failure modes.
• Collaborate with domain experts, engineers, and external partner firms to ensure quality outcomes.
• Make autonomous product decisions and own product outcomes.

Skills

Product management

AI product development

Cross-team collaboration

Delivery leadership

Technical fluency in LLMs

Tools

LLMs

RAG

Embeddings

Job description

• Own agents, the agent builder, the evaluation builder, and agentic memory as products, including their roadmaps, sequencing, and business outcomes
• Own the applied-science delivery roadmap and cross-team orchestration for the Caseware AI platform
• Sequence evaluation, agentic-memory, and platform workstreams into a single delivery plan
• Own the product acceptance bar for every AI feature and establish evaluation baselines
• Work with domain experts and partner accounting and audit firms to validate AI outputs against professional standards
• Serve as the practitioner voice and SME proxy by shadowing audit workflows, testing prototypes, and identifying actionable failure modes
• Partner daily with engineering to convert problem statements into requirements and acceptance criteria
• Monitor AI model developments, architectures, agent patterns, and competitor activity
• Define and own product health metrics, including task completion, correction rates, override frequency, trust signals, and time-to-completion
• Collaborate with applied scientists, engineers, domain experts, and external partner firms
• Make autonomous product decisions and own product outcomes

Requirements

• 6+ years of experience in technology-focused product management or product development
• 3–4 years working deeply with AI or machine learning–driven products
• Software engineers transitioning into product management are welcome
• Proven ability to drive delivery across multiple technical teams, coordinating dependencies, shared interfaces, and timelines across applied science and platform engineering
• Direct experience building and running evaluation frameworks for production AI products
• Experience defining acceptance bars, running human-in-the-loop and practitioner validation, tracking regressions across releases, and using results to decide what ships
• Technical fluency in LLMs, RAG, prompt engineering, embeddings, and agentic patterns
• High agency and ownership of outcomes
• Intellectual curiosity about AI, including following model releases, reading research, and understanding frontier-lab developments
• Experience representing the voice of customers or practitioners and working with domain experts or subject matter specialists
• Experience defining quality bars in high-stakes, regulated, or professional-grade contexts
• Comfort with ambiguity
• Genuine curiosity about auditing and appreciation for high-quality work
• Willingness to build accounting and audit domain fluency on the job

Core Competencies

Demonstrates expertise in technology-focused product management with a strong emphasis on AI and machine learning-driven products. Capable of defining product health metrics, managing cross-team orchestration, and ensuring high-quality outcomes in regulated environments.

Highest-signal resume keywords

• Product Management
• AI Product Development
• Evaluation Frameworks
• Technical Fluency in LLMs
• Cross-Team Collaboration

Hard Skills

• Product Roadmap Ownership
• AI Feature Evaluation
• Acceptance Criteria Definition
• Human-in-the-Loop Validation
• Prompt Engineering
• Agentic Patterns
• Task Completion Metrics
• Correction Rates
• Override Frequency
• Time-to-Completion

Soft Skills

• High Agency
• Intellectual Curiosity
• Comfort with Ambiguity

Industry Keywords

• Applied Science
• Audit Workflows
• Professional Standards
• Domain Expertise
• Accounting Fluency