Product Manager – Data & ML Evaluation (Enterprise) [UAEN]

Black Pearl Consult; Abu Dhabi; On-site; AED 223,200 - 334,800; Full time

Job summary

An innovative consulting firm in Abu Dhabi is seeking a highly analytical Product Manager – Data & ML Evaluation to drive product data strategies and manage machine learning evaluations. This role requires 5–8+ years in product management, expertise in SQL, and proven experience in data analytics frameworks. You will collaborate with cross-functional teams to ensure AI-driven products are measurable and continuously improve, working in a fast-paced environment. Strong stakeholder management skills and a knack for A/B testing are crucial for success.

Qualifications

• 5–8+ years of experience in Product Management with a strong background in data and analytics.
• Proven experience owning product analytics and instrumentation.
• Excellent communication skills across technical and business teams.

Responsibilities

• Own and manage the end-to-end product data ecosystem.
• Design and own machine learning evaluation frameworks.
• Drive data-informed pricing strategies.

Skills

Product Management

Data Analysis

SQL

Stakeholder Management

A/B Testing

Machine Learning

Tools

Amplitude

MLflow

dbt

Job description

Our client is seeking a highly analytical and technically strong Product Manager – Data & ML Evaluation to lead the strategy and execution of product data, experimentation, and machine learning evaluation.

In this role, you will act as the product owner for data and evaluation frameworks, ensuring that AI‑driven products are measurable, reliable, and continuously improving. You will work cross‑functionally with Product, Engineering, Data Science, and business stakeholders to deliver high‑impact, data‑driven solutions.

Key Responsibilities

• Product Data & Analytics
  • Own and manage the end‑to‑end product data ecosystem, including telemetry, tracking plans, event schemas, data quality, and warehousing
  • Define and implement a product metrics framework (north star metrics, leading indicators, input metrics)
  • Enable stakeholders with accessible dashboards and actionable insights across products and platforms
• ML Evaluation & Experimentation
  • Design and own machine learning evaluation frameworks, including test set curation and KPI definition (quality, safety, bias, latency, cost)
  • Run and optimize offline and online evaluations for AI models and features
  • Build scalable and reproducible evaluation pipelines integrated with CI/CD workflows
  • Lead A/B testing and experimentation strategies, including designing, analyzing, and scaling testing frameworks
• Strategy & Governance
  • Drive data‑informed pricing strategies by analyzing usage patterns, feature value, and cost structures
  • Establish and enforce data governance standards, including event naming, privacy, PII handling, and data retention policies
  • Collaborate with Security, Legal, and business teams to ensure compliance and best practices

Requirements

• 5–8+ years of experience in Product Management with a strong background in data, analytics, or engineering
• Proven experience owning product analytics, instrumentation, and experimentation frameworks
• Strong proficiency in SQL and data analysis, with the ability to validate and interpret experiments independently
• Hands‑on experience working with machine learning products, including model evaluation and monitoring
• Strong understanding of A/B testing methodologies and statistical concepts (e.g., power, MDE, guardrail metrics)
• Excellent stakeholder management and communication skills across technical and business teams
• Ability to operate effectively in fast‑paced, cross‑functional environments

Preferred Qualifications

• Experience with product analytics tools (e.g., Amplitude or similar) and experiment tracking platforms (e.g., MLflow)
• Exposure to LLM / RAG evaluation frameworks, safety metrics, and AI model testing methodologies
• Familiarity with data modeling tools (e.g., dbt), feature stores, and observability platforms
• Experience working in regulated or privacy‑sensitive environments
• Comfort leveraging AI tools to prototype analyses, dashboards, or lightweight data utilities

Work Location

• Abu Dhabi