Senior Product Manager

Pagos Consultants; San Francisco (CA); On-site; USD 150,000 - 260,000; Full time

Job summary

Pagos Consultants is seeking a Senior Product Manager with hands-on experience building with large language models and agentic systems to lead AI-powered enterprise products.

This highly technical role sits between product management, applied AI, and software engineering, shaping how intelligent agents reason about complex operational problems, what information they receive, and which tools they can access.

Qualifications

• 3+ years of product management experience on technically sophisticated software products.
• Direct experience building with LLMs rather than solely managing AI vendors or consuming third-party AI functionality.
• Experience shipping production AI features, agent workflows, model-powered applications, or similarly complex AI systems.
• Strong understanding of modern foundation models, including their capabilities, limitations, reliability characteristics, and cost/latency trade-offs.
• Practical knowledge of agent architecture, including tool calling, retrieval, context management, orchestration, memory, and guardrails.
• Experience designing or working with evaluation methodologies for probabilistic and non-deterministic software.
• Ability to understand traces and diagnose why an AI workflow behaved incorrectly.
• Strong judgment around when AI meaningfully improves a product and when deterministic software or human workflows remain the better solution.
• Experience partnering closely with engineers and making technically informed product trade-offs.
• Ability to translate complex technical concepts for customers, executives, and other non-technical stakeholders.
• Experience with technically complex products such as AI infrastructure, developer platforms, APIs, enterprise data products, or data infrastructure.
• Degree in Computer Science, Engineering, Data Science, Information Systems, or another technical discipline, or equivalent practical experience.

Responsibilities

• Set the product strategy and roadmap for AI and agent-based capabilities across the platform.
• Determine which workflows should be fully automated, which should remain human-assisted, and where AI should not be introduced.
• Establish the autonomy model for agents, including which actions can happen independently, which require approval, and how those permissions evolve as reliability improves.
• Work closely with engineering on agent architecture, context construction, retrieval, tool interfaces, orchestration, guardrails, and production deployment.
• Translate customer problems, technical constraints, and emerging model capabilities into detailed product requirements and executable specifications.
• Assess the feasibility and failure modes of new AI capabilities before committing engineering resources.
• Create a rigorous evaluation framework for agent performance, including representative test scenarios, benchmark datasets, quality criteria, regression testing, and production feedback loops.
• Reproduce realistic enterprise environments and operational scenarios to understand how agents behave outside controlled demonstrations.
• Define and track meaningful AI product metrics across areas such as task completion, accuracy, user acceptance, investigation time, automation rate, latency, and cost per successful outcome.
• Analyze failed agent runs and work with engineering to determine whether failures originate from retrieval, context, prompting, tools, orchestration, model behavior, or product design.
• Balance model quality against inference cost, latency, reliability, security, and user experience.
• Design AI experiences that make agent reasoning and actions understandable, reviewable, and correctable by users.
• Partner with commercial and customer-facing teams on launches, technical enablement, product positioning, and explaining agent behavior to customers.
• Track developments across foundation models, agent architectures, tool-use protocols, retrieval techniques, evaluation methodologies, and emerging interoperability standards.

Skills

LLMs experience

AI feature shipping

Agent workflows

Evaluation methodologies

Engineering collaboration

Education

BS in Computer Science, Engineering, Data Science, or related technical field

Tools

Agent frameworks

Tool calling

MCP / interoperability

Retrieval & context engineering

Observability

Job description

We are looking for a Senior Product Manager with meaningful hands-on experience building with large language models and agentic systems to lead the development of a new generation of AI-powered enterprise products.

This is a highly technical product role sitting between product management, applied AI, and software engineering. You will determine how intelligent agents reason about complex operational problems, what information they receive, which tools they can access, what actions they can take independently, and where human judgment remains essential.

You will work across areas including context and retrieval, agent orchestration, tool use, automated investigation and remediation, and integrations with the systems customers already use to manage their day-to-day work.

This is not a strategy-only AI role. You should understand modern model capabilities well enough to diagnose why an agent failed, whether the issue sits in retrieval, context construction, tool design, orchestration, prompting, or the underlying model.

You will use that technical understanding to shape product direction, create detailed requirements, and work alongside engineering to build AI systems that customers can trust with increasingly important workflows.

About the Company

We are a venture-backed enterprise software company building AI-powered infrastructure for modern data and technology teams.

Our platform helps organizations identify operational issues, understand their underlying causes, and automate increasingly complex investigation and resolution workflows. The broader vision is to move enterprise operations away from passive monitoring and manual troubleshooting toward intelligent systems capable of taking meaningful action.

The company is entering a significant growth phase following substantial institutional funding and is backed by leading technology investors.

The team combines experienced startup operators with engineers and product leaders who have previously worked across major cloud, data infrastructure, fintech, and technology companies. The environment is highly technical, fast-moving, and built around strong individual ownership.

What You'll Own

• Set the product strategy and roadmap for AI and agent-based capabilities across the platform.
• Determine which workflows should be fully automated, which should remain human-assisted, and where AI should not be introduced.
• Establish the autonomy model for agents, including which actions can happen independently, which require approval, and how those permissions evolve as reliability improves.
• Work closely with engineering on agent architecture, context construction, retrieval, tool interfaces, orchestration, guardrails, and production deployment.
• Translate customer problems, technical constraints, and emerging model capabilities into detailed product requirements and executable specifications.
• Assess the feasibility and failure modes of new AI capabilities before committing engineering resources.
• Create a rigorous evaluation framework for agent performance, including representative test scenarios, benchmark datasets, quality criteria, regression testing, and production feedback loops.
• Reproduce realistic enterprise environments and operational scenarios to understand how agents behave outside controlled demonstrations.
• Define and track meaningful AI product metrics across areas such as task completion, accuracy, user acceptance, investigation time, automation rate, latency, and cost per successful outcome.
• Analyze failed agent runs and work with engineering to determine whether failures originate from retrieval, context, prompting, tools, orchestration, model behavior, or product design.
• Balance model quality against inference cost, latency, reliability, security, and user experience.
• Design AI experiences that make agent reasoning and actions understandable, reviewable, and correctable by users.
• Partner with commercial and customer-facing teams on launches, technical enablement, product positioning, and explaining agent behavior to customers.
• Track developments across foundation models, agent architectures, tool-use protocols, retrieval techniques, evaluation methodologies, and emerging interoperability standards.

What We're Looking For

Core Requirements

• 3+ years of product management experience working on technically sophisticated software products.
• Direct experience building with LLMs rather than solely managing AI vendors or consuming third-party AI functionality.
• Experience shipping production AI features, agent workflows, model-powered applications, or similarly complex AI systems.
• Strong understanding of modern foundation models, including their capabilities, limitations, reliability characteristics, and cost/latency trade-offs.
• Practical knowledge of agent architecture, including tool calling, retrieval, context management, orchestration, memory, and guardrails.
• Experience designing or working with evaluation methodologies for probabilistic and non-deterministic software.
• Ability to understand traces and diagnose why an AI workflow behaved incorrectly.
• Strong judgment around when AI meaningfully improves a product and when deterministic software or human workflows remain the better solution.
• Experience partnering closely with engineers and making technically informed product trade-offs.
• Ability to translate complex technical concepts for customers, executives, and other non-technical stakeholders.
• Experience with technically complex products such as AI infrastructure, developer platforms, APIs, enterprise data products, or data infrastructure.
• Degree in Computer Science, Engineering, Data Science, Information Systems, or another technical discipline, or equivalent practical experience.

Particularly Relevant Experience

We are especially interested in candidates who have worked with one or more of the following:

• Agent frameworks and multi-step LLM workflows
• Tool calling and structured model outputs
• Model Context Protocol (MCP) or similar interoperability approaches
• Retrieval and context engineering
• Agent evaluation and observability
• Developer tooling or technical B2B SaaS
• AI integrations into collaboration, ticketing, or workflow-management platforms
• High-growth startup environments

The Type of Product Manager Who Will Thrive Here

You will likely enjoy this role if you like getting underneath how AI systems actually work rather than treating the model as a black box.

You should be comfortable discussing model behavior with engineers, inspecting failed workflows, challenging assumptions about automation, and determining what evidence is required before an agent can be trusted with greater autonomy.

You are likely someone who prototypes with new models and tools, understands the difference between an impressive demo and a dependable production system, and enjoys turning rapidly evolving AI capabilities into reliable enterprise software.