AI Reliability & Evaluation Engineer

Nanyang Technological University Singapore; Singapore; On-site; SGD 90,000 - 130,000; Full time

Job summary

Nanyang Technological University’s National Centre for Research in Digital Trust (DTC) seeks a Research Engineer to advance evaluation and validation of AI systems. You will design evaluation methods, build test datasets, and analyze model performance to guide improvements and tooling development.

Ideal candidates have a solid CS/engineering background, strong Python skills, and hands-on NLP/LLMs experience. Collaboration with partners and clear communication to varied audiences are essential.

Qualifications

• Proficiency in Python and practical experience with ML workflows including model evaluation and error analysis.
• Working knowledge of NLP and hands-on experience with large language models.
• Experience in evaluation methodology or data annotation will be a strong advantage.
• Careful, methodical approach and ability to work independently under tight timelines.

Responsibilities

• Work directly with project partners and clients — participating in requirement discussions and technical meetings, preparing materials, and following up on action items.
• Design evaluation frameworks, test datasets, and scoring methodology for AI and ML systems.
• Develop annotation guidelines and monitor labelling quality.
• Conduct error analysis and translate findings into clear recommendations for improvement.
• Run structured experiments with large language models, and build supporting prototypes and tooling.
• Communicate results clearly to both technical and non-technical audiences.

Skills

Python

ML workflows

Model evaluation

Error analysis

NLP

Large language models

Education

Bachelor’s degree in computer science, engineering, data science, or a related field

Job description

Nanyang Technological University’s National Centre for Research in Digital Trust (DTC) seeks a Research Engineer to advance evaluation and validation of AI systems. You will design evaluation methods, build test datasets, and analyze model performance to guide improvements and tooling development.

Ideal candidates have a solid CS/engineering background, strong Python skills, and hands-on NLP/LLMs experience. Collaboration with partners and clear communication to varied audiences are essential.