Our client, one of Asia-Pacific’s leading organizations is looking for:
Data Scientist
Responsibilities
- Evaluate models and methodology: Review AI use cases, data suitability, modelling assumptions, validation methods and performance measures
- Conduct reproducible tests of accuracy, reliability, robustness, fairness and explainability, with findings documented for each assigned assessment
- Assess AI risks and safeguards: Assess risk levels using applicable government and agency frameworks
- For LLM and agentic systems, test relevant risks such as hallucination, prompt injection, sensitive data disclosure and unauthorised actions; recommend safeguards and assess residual risks
- Translate applicable IM8 and agency requirements into practical checklists, assessment templates and workflows
- Develop evaluation and monitoring tools: Adapt reusable Python scripts, test sets with project teams and engineers
- Provide monthly updates on cases, risks, outstanding actions and delivery progress. Guide project teams on assessment requirements and hand over evaluation code
Requirements
- Degree in Data Science, Computing, Statistics, Mathematics, Engineering or a related with minimum 3 years of relevant experience in data science, machine learning or AI, including hands-on model evaluation and validation
- Experience with end-to-end data science projects, assessing technical limitations and communicating findings to stakeholders.
- Strong technical understanding of LLMs, especially how they work, how to do prompt engineering effectively, and how to build apps with them
- Good understanding of key AI safety and/or algorithmic fairness concepts, especially in how to operationalise and apply them to AI systems
- Proficient in Python programming, with some familiarity of cloud deployments for ML models and LLMs
- Ability to apply RAI and risk assessment principles, assess policy compliance, document evidence and residual risks, and manage governance submissions and follow-up actions
- Experience with LLM safety testing, prompt injection testing, fairness assessments, explainability; Familiarity with IM8 and the principles behind its requirements
EA License Number: 10C3636