Experience: 3+ Years in Product Management
Location:SF Hybrid
About the Role
We are seeking a technically strong Senior Product Manager (AI) with hands-on experience building and deploying agentic AI systems to lead the strategy and execution of our AI product portfolio.
This role sits at the intersection of product leadership and applied AI engineering. You will define how AI agents identify, investigate, and resolve enterprise data incidents, including the context they can access, the actions they can perform autonomously, how their outputs are evaluated, and when human intervention is required.
You will work across context management, incident detection, triage and resolution agents, and integrations that bring AI capabilities into the tools customers already use, such as Microsoft Teams, Slack, ServiceNow, and Jira.
The ideal candidate combines strong product management experience with a deep technical understanding of LLMs, agent architectures, retrieval, context engineering, tool calling, and AI evaluation. You should be comfortable analysing failed agent traces, identifying underlying issues, and translating technical insights into product decisions.
Requirements
Key Responsibilities
- Own the AI product strategy, roadmap, and prioritisation, defining which workflows agents should manage independently, where they should assist users, and where human oversight is necessary.
- Evaluate the feasibility, limitations, failure modes, and long-term maintainability of agentic AI features.
- Define agent autonomy and human-in-the-loop boundaries, gradually expanding autonomous capabilities as reliability and user trust improve.
- Translate customer requirements, technical constraints, and model capabilities into detailed Product Requirement Documents (PRDs) and actionable engineering specifications.
- Partner closely with engineering teams to design, develop, evaluate, and launch production-ready AI agents.
- Establish quality standards across accuracy, latency, cost, observability, security, and reliability.
- Build and maintain AI evaluation frameworks, including golden datasets, quality rubrics, and regression testing processes.
- Replicate real-world customer environments and incident scenarios to validate agent performance and production readiness.
- Define and monitor AI product success metrics, including resolution accuracy, acceptance rates, time to root cause, deflection rates, and token cost per outcome.
- Analyse production data and customer feedback to identify performance gaps and continuously improve AI capabilities.
- Collaborate with Go-to-Market teams on product launches, technical enablement, and customer-facing documentation.
- Communicate agent capabilities, limitations, context sources, and decision-making processes clearly to customers.
- Stay updated on foundation models, agent frameworks, Model Context Protocol (MCP), tool-calling standards, retrieval techniques, and AI evaluation methodologies.
- Champion a transparent, reliable, and user-friendly experience for data engineers and analysts.
Required Qualifications
- 3+ years of product management experience working on technically complex products, such as AI/ML platforms, data platforms, developer tools, or APIs.
- Hands-on experience building with Large Language Models (LLMs) in a technical role, including shipping production AI features, developing agent workflows, prompt and context engineering, or conducting model evaluations.
- Strong understanding of foundation model capabilities, limitations, and cost, latency, and quality trade-offs.
- Working knowledge of agent architecture patterns, including tool calling, context assembly, retrieval, orchestration, and guardrails.
- Experience with evaluation methodologies for non-deterministic AI systems.
- Strong product judgment and the ability to identify when AI is an appropriate solution and when alternative approaches are more suitable.
- Proven ability to collaborate closely with engineering teams and make informed technical and product trade-offs.
- Excellent communication skills, with the ability to explain complex technical concepts to technical and non-technical stakeholders.
- Bachelor's degree in Computer Science, Data Science, Information Systems, Engineering, or a related technical field, or equivalent practical experience.
Preferred Qualifications
- Experience with MCP, agent frameworks, or integrating AI capabilities into platforms such as Teams, Slack, ServiceNow, and Jira.
- Previous ownership of an AI or agent-based product within a B2B SaaS or data-focused organisation.
- Familiarity with data observability, data lineage, governance, and metadata management platforms.
- Background in data engineering or data architecture, or experience building AI solutions on enterprise data systems.
- Experience working in a high-growth SaaS or developer-focused product environment.
Ideal Candidate Profile
This role is suited to a product leader who has practical experience building with AI models rather than limiting their involvement to strategy or vendor evaluation. You should enjoy working through the technical challenges of agentic systems, particularly improving their accuracy, reliability, and ability to earn user trust.