Senior AI Evaluation Platform Product Manager

Arize AI; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Arize AI is seeking a Product Manager to own the Evals & Annotations product area end-to-end in the United States. You will define the roadmap, balance customer needs with platform investments, and partner with Design to craft an exceptional evaluation experience for AI and agent development.

You will shape core product experiences, prototype solutions, and drive UX improvements, while aligning with a fast-paced startup culture. Strong understanding of LLM evaluations is essential.

Qualifications

• Significant product management experience independently owning technically complex developer-facing products.
• Strong understanding of LLM evaluations and AI development workflows, including automated evals and human feedback.
• Excellent product and UX instincts, with the ability to define and prototype core product experiences.

Responsibilities

• Own Arize's Evals & Annotations product area end-to-end, including eval creation and execution, human review, annotation workflows, and the broader evaluation lifecycle.
• Set the product strategy and roadmap, balancing customer needs, foundational platform investments, UX improvements, and emerging opportunities in AI and agent development.
• Drive a best-in-class evaluation experience by developing strong opinions, prototyping solutions, and partnering with Design.

Skills

Product management

LLM evaluations

Developer-facing

UX instincts

Prototyping

Job description

Arize AI is seeking a Product Manager to own the Evals & Annotations product area end-to-end in the United States. You will define the roadmap, balance customer needs with platform investments, and partner with Design to craft an exceptional evaluation experience for AI and agent development.

You will shape core product experiences, prototype solutions, and drive UX improvements, while aligning with a fast-paced startup culture. Strong understanding of LLM evaluations is essential.