Lead AI Product Manager, GenAI & Agentic Systems

Robots and Pencils; United States; Remote; USD 44 - 70; Full time

Benefits offered by this job

Time off

Medical/Dental/Vision

401(k)

Job summary

Robots & Pencils is seeking a Senior Product Manager, AI to lead end-to-end delivery for GenAI/agentic AI engagements. You will own the roadmap, define agent behavior, run evals, and guide discovery through phased rollout with senior client stakeholders.

You will work with cross-functional teams, framing AI tradeoffs for executives, prototyping with Claude/Cursor/GitHub Copilot, and ensuring production-ready solutions within enterprise settings.

Qualifications

• 5+ years of experience in product management with hands-on experience shipping AI features or products.
• Ability to read code (Python, JavaScript, or TypeScript), evaluate architecture decisions, and make informed product tradeoffs on technical constraints
• Deep understanding of AI concepts including LLMs, RAG, fine‑tuning, prompt and context engineering, agentic systems such as planning, tool use, human‑in‑the‑loop controls, and guardrails, and evals
• Proven ability to prototype AI and agentic solutions using modern AI tools to validate hypotheses and de‑risk product decisions (e.g., Claude, Claude Code, Cursor, GitHub Copilot)
• Experience running evals for AI systems and iterating based on results
• Experience defining agent behavior including system prompts, tool schemas, and human‑in‑the‑loop boundaries
• Strong experience working with AWS AI services in a product context (e.g., Bedrock, Lambda, SageMaker, Kendra)
• Strong client communication skills and the ability to frame AI capabilities and limitations for non‑technical executive audiences
• Comfortable operating in ambiguous, fast‑moving environments where the AI landscape evolves rapidly
• Experience in a consulting or agency environment is a plus.
• Familiarity with agent orchestration frameworks (LangChain, LangGraph, AWS Strands) is a plus.
• Software engineering background (Python, JavaScript, TypeScript) is a plus.

Responsibilities

• Own end‑to‑end product delivery for complex AI and agentic AI engagements, from discovery and hypothesis framing through phased rollout and production readiness
• Define and maintain AI product roadmaps that connect agent capabilities to measurable enterprise business outcomes
• Lead discovery activities including user research, stakeholder interviews, and assumption testing — framing the riskiest hypotheses first and de‑risking them before committing to build
• Design and run experiments including POCs, pilot deployments, and scenario‑based testing of multi‑step agentic workflows, edge cases, and failure recovery
• Define agent behavior end‑to‑end including system prompts, tool schemas, context management, summarization, and working memory patterns across multi‑step tasks
• Own backlog management and sprint execution day‑to‑day on complex AI engagements — prioritizing work, refining agent behavior and requirements with engineering, and keeping the team moving.
• Own and run evals frameworks covering task completion, reasoning quality, tool selection, failure recovery, and safety — iterating based on evidence rather than assumptions
• Define success metrics at the agent level including task completion rate, cost per task, escalation rate, and time to resolution alongside business KPIs
• Establish human‑in‑the‑loop boundaries and guardrails — when the agent acts autonomously, when it escalates, and how to handle non‑deterministic output
• Bring an AI‑forward mindset to your daily work, using tools like Claude, Claude Code, Cursor, and GitHub Copilot — and agentic frameworks where relevant — to move faster than traditional product

Skills

AI product mgmt

Code literacy

AI concepts

Prototype AI

Evals & metrics

Agent behavior design

AWS AI services

Client communication

Ambiguity tolerance

Consulting experience

LangChain familiarity

Open-source LLMs

Software eng background

Tools

AWS Bedrock

AWS Lambda

SageMaker

Kendra

Job description

Robots & Pencils is seeking a Senior Product Manager, AI to lead end-to-end delivery for GenAI/agentic AI engagements. You will own the roadmap, define agent behavior, run evals, and guide discovery through phased rollout with senior client stakeholders.

You will work with cross-functional teams, framing AI tradeoffs for executives, prototyping with Claude/Cursor/GitHub Copilot, and ensuring production-ready solutions within enterprise settings.