Remote Product Manager — AI Evaluations Platform

Relativity ODA LLC; United States; Hybrid; USD 150,000 - 210,000; Full time

Job summary

Relativity is seeking a Product Manager to own the strategy and roadmap for the Evaluations Platform, enabling AI product teams to measure, understand, and improve AI performance. You will drive continuous discovery with scientists, engineers, and PMs, shape the agentic testing system, and ensure fast, safe deployment of capabilities at scale.

You will partner with legal and product stakeholders to define evaluation tasks, establish error criteria, and maintain dashboards that surface product

Qualifications

• 4+ years of product management at a technology company.
• 2+ years on technical, platform, infrastructure, or data/ML products.
• Strong understanding of the software development lifecycle and agile practices.
• Experience conducting user discovery with technical audiences and translating findings into roadmap decisions.
• Excellent written and verbal communication skills, with the ability to explain complex technical concepts to both technical and non-technical audiences.

Responsibilities

• Own the product vision, strategy, and roadmap for Relativity's Evaluations Platform across its pillars.
• Run continuous discovery with applied scientists, application engineering teams, and PMs to measure AI quality.
• Lead the agentic testing system and ship new capabilities safely and quickly.
• Define SME tasks, build authoring/approval workflows for experts.
• Own team-facing quality dashboards to show performance and changes since last release.
• Lead build-vs-buy evaluations for tracing infrastructure and eval tooling.
• Collaborate with engineering and applied science on judge calibration for gating deployments.
• Clearly articulate technical tradeoffs to stakeholders across levels.

Skills

LLM-based products

Platform/DevTools PM

Technical customers

Agile practices

User discovery

Clear communication

Job description

Relativity is seeking a Product Manager to own the strategy and roadmap for the Evaluations Platform, enabling AI product teams to measure, understand, and improve AI performance. You will drive continuous discovery with scientists, engineers, and PMs, shape the agentic testing system, and ensure fast, safe deployment of capabilities at scale.

You will partner with legal and product stakeholders to define evaluation tasks, establish error criteria, and maintain dashboards that surface product