AI Evaluation Engineer - Trustworthy LLMs & Tools

Aimlroles; Singapore; On-site; SGD 60,000 - 90,000; Full time

Job summary

The National Centre for Research in Digital Trust (DTC) at NTU seeks a Research Engineer to support evaluation and validation of AI systems. You will design evaluation methods, build test datasets, and analyze performance in real-world conditions.

You will collaborate with external partners, run experiments with large language models, develop prototypes and tooling, and communicate results to both technical and non-technical audiences.

Qualifications

• Bachelor’s degree in computer science, engineering, data science, or a related field.
• Confidence to engage with external partners and explain technical concepts clearly to non-technical audiences.
• Proficiency in Python, with practical experience in ML workflows including model evaluation and error analysis.
• Working knowledge of NLP and hands-on experience with large language models.
• Experience in evaluation methodology or data annotation will be a strong advantage.
• Careful, methodical, and able to work independently under tight timelines.

Responsibilities

• Design evaluation frameworks, test datasets, and scoring methodology for AI and ML systems.
• Develop annotation guidelines and monitor labelling quality.
• Conduct error analysis and translate findings into clear recommendations for improvement.
• Run structured experiments with large language models, and build supporting prototypes and tooling.
• Communicate results clearly to both technical and non-technical audiences.

Skills

Python

NLP

ML evaluation

Error analysis

Annotation guidelines

Communication

Education

Bachelor's degree in CS/Engineering/Data Science

Job description

The National Centre for Research in Digital Trust (DTC) at NTU seeks a Research Engineer to support evaluation and validation of AI systems. You will design evaluation methods, build test datasets, and analyze performance in real-world conditions.

You will collaborate with external partners, run experiments with large language models, develop prototypes and tooling, and communicate results to both technical and non-technical audiences.