Senior Product Manager, AI & Agent Systems

Pagos Consultants; San Francisco (CA); On-site; USD 150,000 - 260,000; Full time

Job summary

Pagos Consultants is seeking a Senior Product Manager with hands-on experience building with large language models and agentic systems to lead AI-powered enterprise products.

This highly technical role sits between product management, applied AI, and software engineering, shaping how intelligent agents reason about complex operational problems, what information they receive, and which tools they can access.

Qualifications

• 3+ years of product management experience on technically sophisticated software products.
• Direct experience building with LLMs rather than solely managing AI vendors or consuming third-party AI functionality.
• Experience shipping production AI features, agent workflows, model-powered applications, or similarly complex AI systems.
• Strong understanding of modern foundation models, including their capabilities, limitations, reliability characteristics, and cost/latency trade-offs.
• Practical knowledge of agent architecture, including tool calling, retrieval, context management, orchestration, memory, and guardrails.
• Experience designing or working with evaluation methodologies for probabilistic and non-deterministic software.
• Ability to understand traces and diagnose why an AI workflow behaved incorrectly.
• Strong judgment around when AI meaningfully improves a product and when deterministic software or human workflows remain the better solution.
• Experience partnering closely with engineers and making technically informed product trade-offs.
• Ability to translate complex technical concepts for customers, executives, and other non-technical stakeholders.
• Experience with technically complex products such as AI infrastructure, developer platforms, APIs, enterprise data products, or data infrastructure.
• Degree in Computer Science, Engineering, Data Science, Information Systems, or another technical discipline, or equivalent practical experience.

Responsibilities

• Set the product strategy and roadmap for AI and agent-based capabilities across the platform.
• Determine which workflows should be fully automated, which should remain human-assisted, and where AI should not be introduced.
• Establish the autonomy model for agents, including which actions can happen independently, which require approval, and how those permissions evolve as reliability improves.
• Work closely with engineering on agent architecture, context construction, retrieval, tool interfaces, orchestration, guardrails, and production deployment.
• Translate customer problems, technical constraints, and emerging model capabilities into detailed product requirements and executable specifications.
• Assess the feasibility and failure modes of new AI capabilities before committing engineering resources.
• Create a rigorous evaluation framework for agent performance, including representative test scenarios, benchmark datasets, quality criteria, regression testing, and production feedback loops.
• Reproduce realistic enterprise environments and operational scenarios to understand how agents behave outside controlled demonstrations.
• Define and track meaningful AI product metrics across areas such as task completion, accuracy, user acceptance, investigation time, automation rate, latency, and cost per successful outcome.
• Analyze failed agent runs and work with engineering to determine whether failures originate from retrieval, context, prompting, tools, orchestration, model behavior, or product design.
• Balance model quality against inference cost, latency, reliability, security, and user experience.
• Design AI experiences that make agent reasoning and actions understandable, reviewable, and correctable by users.
• Partner with commercial and customer-facing teams on launches, technical enablement, product positioning, and explaining agent behavior to customers.
• Track developments across foundation models, agent architectures, tool-use protocols, retrieval techniques, evaluation methodologies, and emerging interoperability standards.

Skills

LLMs experience

AI feature shipping

Agent workflows

Evaluation methodologies

Engineering collaboration

Education

BS in Computer Science, Engineering, Data Science, or related technical field

Tools

Agent frameworks

Tool calling

MCP / interoperability

Retrieval & context engineering

Observability

Job description

Pagos Consultants is seeking a Senior Product Manager with hands-on experience building with large language models and agentic systems to lead AI-powered enterprise products.

This highly technical role sits between product management, applied AI, and software engineering, shaping how intelligent agents reason about complex operational problems, what information they receive, and which tools they can access.