AI Product Analyst: Shape Trustworthy AI Tools

IMDA; Singapore; On-site; SGD 60,000 - 90,000; Part time

Job summary

AI Verify Foundation, a wholly owned subsidiary under IMDA, is seeking a Product Analyst to elevate the quality and value of our product offerings for developers and data scientists. You’ll bridge cutting-edge AI research with real-world evaluation needs, shaping tools and frameworks for the success of our community, and lead QA of datasets and evaluators.

You will design offline A/B experiments to optimize evaluation pipelines for Moonshot benchmarks, contribute to Singapore-specific test

Qualifications

• Background in Statistics or Data Science, or a related technical field.
• 1-3 years’ experiences as a data scientist, with strong grounding in experimental design and analysis.
• Proficiency in Python/ R is required. Experience in AI testing is a plus.
• Deep intellectual capacity combined with a practical, user-centric mindset.
• Excellent communication skills— excel in storytelling with data, and able to simplify complex concepts for varied audiences.

Responsibilities

• Design and run offline A/B experiments to design optimal evaluation pipelines for benchmark tests offered in Project Moonshot, including LLM-as jury.
• Contribute to the development and maintenance of benchmark datasets, particularly generating realistic test cases in the context of Singapore.
• Research emerging GenAI evaluation best practices and open-source tools — translate key insights to inform product roadmap, and turn research findings into actionable prototypes for engineering teams.
• Educate and engage the developer community to build awareness of the challenges in benchmark testing, and champion the solutions provided in our library.

Skills

Python

R

Experimental design

Data storytelling

Communication

Education

Statistics or Data Science

Tools

Python

R

Job description

AI Verify Foundation, a wholly owned subsidiary under IMDA, is seeking a Product Analyst to elevate the quality and value of our product offerings for developers and data scientists. You’ll bridge cutting-edge AI research with real-world evaluation needs, shaping tools and frameworks for the success of our community, and lead QA of datasets and evaluators.

You will design offline A/B experiments to optimize evaluation pipelines for Moonshot benchmarks, contribute to Singapore-specific test