AI Product Manager - Evaluation & Data

Rakuten Global; Singapore; On-site; SGD 120,000 - 170,000; Full time

Job summary

Rakuten Global in Singapore seeks a Product Manager – Evaluation and Data to make AI product quality measurable and actionable. You will own the evaluation strategy, process, and platform to enable teams to define success, prevent regressions, and improve AI experiences through reliable data and insights.

You will collaborate with product, engineering, data science, analytics, and business teams to align success criteria with production monitoring, A/B testing, and KPI-driven releases.

Qualifications

• 5+ years of product management experience in AI/ML, evaluation, data products, or platforms.
• Excellent analytical skills and experience working with large datasets.
• Ability to define platform requirements and communicate model/prompt versioning, tracing, data pipelines, and release integration.
• Experience with A/B testing, production monitoring, user feedback, and human evaluation.
• Excellent written and verbal communication, with the ability to present complex evaluation results clearly and concisely.
• Demonstrated ability to lead cross-functional product, engineering, data science, and domain teams.

Responsibilities

• Define success for Rakuten's AI products and translate requirements, user needs, and risks into measurable criteria.
• Calibrate automated graders with human judgment and review transcripts and failures.
• Define the platform vision, roadmap, and requirements with engineering and data teams.
• Prioritize dataset management, model/prompt comparison, automated and human grading, multi-turn evaluation, and dashboards.
• Establish governance and connect evaluations with production monitoring, A/B testing, feedback, and business KPIs.

Skills

PM experience in AI/ML

Data analytics

Platform requirements

Experimentation & QA

Communication skills

Cross-functional collaboration

Tools

A/B testing tools

Analytics dashboards

Job description

Rakuten Global in Singapore seeks a Product Manager – Evaluation and Data to make AI product quality measurable and actionable. You will own the evaluation strategy, process, and platform to enable teams to define success, prevent regressions, and improve AI experiences through reliable data and insights.

You will collaborate with product, engineering, data science, analytics, and business teams to align success criteria with production monitoring, A/B testing, and KPI-driven releases.