AI Evaluation Product Analyst for Open-Source Benchmarks

sggovterp; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

AI Verify Foundation in Singapore is seeking a Product Analyst to bridge cutting-edge AI research with real-world evaluation needs. You will lead quality assurance of datasets and evaluators in our products, shaping tools for our developer community.

Responsibilities include designing offline experiments, developing benchmark datasets for Singapore, and translating research insights into actionable prototypes for engineering teams. Excellent communication is essential.

Qualifications

• Background in Statistics or Data Science.
• 1–3 years' experience as a data scientist.
• Proficiency in Python or R.
• Strong communication with data storytelling.

Responsibilities

• Design and run offline A/B experiments to optimize evaluation pipelines.
• Contribute to benchmark datasets and generate realistic test cases for Singapore.
• Research GenAI evaluation practices and translate insights to product roadmap.
• Educate and engage the developer community about benchmark testing.

Skills

Data analysis

Experiment design

Storytelling with data

Communication

Education

Statistics or Data Science

Tools

Python

R

Job description

AI Verify Foundation in Singapore is seeking a Product Analyst to bridge cutting-edge AI research with real-world evaluation needs. You will lead quality assurance of datasets and evaluators in our products, shaping tools for our developer community.

Responsibilities include designing offline experiments, developing benchmark datasets for Singapore, and translating research insights into actionable prototypes for engineering teams. Excellent communication is essential.