Lead Product Manager, AI Agent Runtime & Evaluations

Juniper Square; Dallas (TX); Remote; USD 200,000 - 230,000; Full time

Benefits offered by this job

Health, dental, vision

Life insurance

Mental wellness coverage

Fertility support

Flex Time Off

Paid family leave

Retirement plans

Home office stipend

Professional development

Job summary

Juniper Square is seeking a Lead Product Manager to own the agent runtime and evaluation layer of Foundry—the platform that builds, deploys, and operates AI agents, apps, and code for fund administration. You will lead non‑engineering users to governed automation with clear identity, access, and audit trails.

The role emphasizes AI fluency, prototyping with LLMs, and collaboration with engineering to ship robust agent‑driven workflows in a fast‑paced environment.

Qualifications

• 7+ years of product management experience in B2B or Enterprise SaaS, including 3+ years in an AI‑oriented role.
• Proven experience owning a complex product area end‑to‑end, from strategy and roadmap through execution and iteration, ideally in a zero‑to‑one context.
• Hands‑on understanding of how agent evaluation works: golden datasets, data labeling, automated judges, regression testing, and knowing when a human has to step in.
• Background building agents and replacing human‑driven processes with product and automation, with real evaluation and monitoring discipline in production.
• A strong engineering partner with a product mindset. You can go deep with engineers on topics like typed contracts between agents and workflows, structured and deterministic outputs, and agent memory.
• User empathy for non‑engineering builders and operators, including how people oversee large numbers of automated agents.
• Fluency with LLMs and common AI tools (e.g. Claude, Cursor). You should be able to run your own prototypes and use AI to get work done faster than a traditional PM workflow would allow.

Responsibilities

• Build the agent runtime for non‑engineers: Own the vision, strategy, and roadmap for a platform where business users take an agent, app, or script from idea to governed production without an engineering cycle.
• Own evaluations and model governance: Define how every automation is tested before it ships. Work with engineering and our eval platform team on golden datasets, data labeling, and regression gates that block promotion.
• Solve the "who evaluates the evaluator" problem: Agents that grade other agents need their own checks and patterns.
• Shape memory, learning, and self‑improvement: Define how agents keep long‑lived memory and improve from experience.
• Run a model‑agnostic platform: Own model routing, cost attribution, and pricing so teams can choose the right model for each job.
• Integrate with broader systems: Define how hosted agents reach data across our product suite and third‑party tools, with a solid connector and access model.
• Give people oversight at scale: Design dashboards, run history, and auditability tools for large fleets of agents.
• Define how automations bind to workflows: Partner on agent registry and contracts that describe needs and outputs.
• Deliver configuration to business SMEs: Build tools for SMEs to configure, test, and tune automations.
• Stay ahead of the market: Track managed runtimes and platforms to inform build/buy/partner decisions.
• Be a leader: Set direction across the platform and mentor others.

Skills

Product management

AI‑driven product

Agent evaluation

Agent development

Cross‑functional leadership

Communication

LLMs

Tools

Claude

LLM tooling

Job description

Juniper Square is seeking a Lead Product Manager to own the agent runtime and evaluation layer of Foundry—the platform that builds, deploys, and operates AI agents, apps, and code for fund administration. You will lead non‑engineering users to governed automation with clear identity, access, and audit trails.

The role emphasizes AI fluency, prototyping with LLMs, and collaboration with engineering to ship robust agent‑driven workflows in a fast‑paced environment.