AI Engineer (Evaluation)

Nanyang Technological University

Singapore

Presencial

SGD 90.000 - 130.000

Jornada completa

14 días+

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Descripción de la vacante

NTU Singapore’s AISG invites a capable researcher to join as an AI evaluation specialist with a focus on multilingual, multicultural, and multimodal LLM evaluation. You will build and maintain evaluation frameworks, collect and translate evaluation data, and collaborate with partners to ensure reliable AI solutions.

You should hold a degree in CS/AI/data science, be proficient in Python with Git, and have experience with PyTorch and vLLM.

Formación

  • Degree in computer science, AI, data science or related field.
  • Deep understanding of LLM evaluation experimental design and the advantages/limitations of various LLM evaluation methods.
  • Experience in LLM inference frameworks such as vLLM and AI/Deep Learning frameworks such as PyTorch.
  • Writing production level code in Python and using version control systems such as Git.
  • Fluent in English and one other Southeast Asian language for multilingual evaluations.

Responsabilidades

  • Develop and maintain evaluation frameworks and pipelines to measure the capabilities of Large Language Models (LLMs).
  • Keep up to date and experiment with the latest research in multilingual, multicultural, and multimodal LLM evaluations such as the LLM-as-a-Judge paradigm.
  • Work with partners to collect, translate and verify evaluation datasets.
  • Perform the necessary data preparation and analysis, AI modelling, coding, testing, validation and deployment to ensure reliable and scalable AI solutions.
  • Collaborate with cross-functional teams within AI Products to design and resolve issues.
  • Maintain code repository and documentation standards.
  • Contribute to community engagement activities such as sharings via technical session meet-ups and article write-ups, and participating in discussion forums.

Conocimientos

Communication skills
Multilingual capability
Independent learning
Research paper literacy
Python programming

Educación

Degree in computer science/AI/data science or related field

Herramientas

vLLM
PyTorch
Git

Descripción del empleo

AI Singapore (AISG) is Singapore's national programme in artificial intelligence, launched by the National Research Foundation (NRF) to anchor deep national capabilities in AI. Hosted at Nanyang Technological University (NTU), AI Singapore brings together Singapore-based research institutions and the vibrant ecosystem of AI start-ups and companies to perform use-inspired research, grow knowledge, create tools, and develop the talent to power Singapore's AI efforts. Since our inception in 2017, we have established a culture of respect, continuous learning, experimentation and curiosity, centred around innovation.

The candidate will join a team of AI scientists, apprentices, data and software engineers. With the team, he or she will be responsible for building evaluations that test the limits of AI models especially in terms of its multilingual, multicultural and multimodal capabilities.

Duties and Responsibilities:
  • Develop and maintain evaluation frameworks and pipelines to measure the capabilities of Large Language Models (LLMs).
  • Keep up to date and experiment with the latest research in multilingual, multicultural, and multimodal LLM evaluations such as the LLM-as-a-Judge paradigm.
  • Work with partners to collect, translate and verify evaluation datasets.
  • Perform the necessary data preparation and analysis, AI modelling, coding, testing, validation and deployment to ensure reliable and scalable AI solutions.
  • Collaborate with cross-functional teams within AI Products to design and resolve issues.
  • Maintain code repository and documentation standards.
  • Contribute to community engagement activities such as sharings via technical session meet-ups and article write-ups, and participating in discussion forums.
Requirements:
  • Graduate from a degree program in computer science, AI, data science or related fields (or equivalent practical experience).
  • Deep understanding of LLM evaluation experimental design and the advantages/limitations of various LLM evaluation methods.
  • Experience in LLM inference frameworks such as vLLM and AI/Deep Learning frameworks such as PyTorch.
  • Writing production level code in Python and using version control systems such as Git.
  • Strong written and verbal communication skills.
  • Independent learner that is capable of reading and understanding research papers.
  • Fluent in English and one other Southeast Asian language for the purposes of understanding how to build quality evaluations from a multilingual and multicultural perspective.

Hiring Institution: NTU A research-intensive public university, Nanyang Technological University, Singapore (NTU Singapore) has 33,000 undergraduate and postgraduate students in the Engineering, Business, Science, Humanities, Arts, & Social Sciences, and Graduate colleges. It also has a medical school, the Lee Kong Chian School of Medicine, established jointly with Imperial College London. NTU is also home to world-class autonomous institutes – the National Institute of Education, S Rajaratnam School of International Studies, Earth Observatory of Singapore, and Singapore Centre for Environmental Life Sciences Engineering – and various leading research centres such as the Nanyang Environment & Water Research Institute (NEWRI) and Energy Research Institute @ NTU (ERI@N). Ranked amongst the world’s top universities by QS, NTU has also been named the world’s top young university for the past seven years. The University’s main campus is frequently listed among the Top 15 most beautiful university campuses in the world and has 57 Green Mark-certified (equivalent to LEED-certified) buildings, of which 95% are certified Green Mark Platinum. Apart from its main campus, NTU also has a campus in Novena, Singapore’s healthcare district. Under the NTU Smart Campus vision, the University harnesses the power of digital technology and tech-enabled solutions to support better learning and living experiences, the discovery of new knowledge, and the sustainability of resources. For more information, visit www.ntu.edu.sg

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Assistant Director, Lead AI Solution Architect – Education Tech Services
Assistant Director, Lead AI Solution Architect – Education Tech Services

Nanyang Technological University • Singapore

Presencial
SGD 120.000 - 170.000
Research Associate (AI in Education)
Research Associate (AI in Education)

Nanyang Technological University • Singapore

Presencial
SGD 67.000 - 100.000
AI Engineer (AIAP)
AI Engineer (AIAP)

Nanyang Technological University • Singapore

Presencial
SGD 90.000 - 150.000
Research Assistant (English Language and Literature Department) [NIE]
Research Assistant (English Language and Literature Department) [NIE]

Nanyang Technological University • Singapore

Presencial
SGD 40.000 - 51.000
AI Engineer (AI Products)
AI Engineer (AI Products)

Nanyang Technological University • Singapore

Presencial
SGD 70.000 - 110.000
Research Fellow (AI in Education)
Research Fellow (AI in Education)

Nanyang Technological University • Singapore

Presencial
SGD 70.000 - 120.000
Research Fellow (AI / Computer Science / Machine Learning / NLP)
Research Fellow (AI / Computer Science / Machine Learning / NLP)

Nanyang Technological University • Singapore

Presencial
SGD 80.000 - 120.000
Research Engineer I (Computer Science / Artificial Intelligence / Software Engineering / Information Systems)
Research Engineer I (Computer Science / Artificial Intelligence / Software Engineering / Information Systems)

Nanyang Technological University • Singapore

Presencial
SGD 60.000 - 90.000
Research Engineer I (Computer Science / Artificial Intelligence / Data Science / Electrical Engineering)
Research Engineer I (Computer Science / Artificial Intelligence / Data Science / Electrical Engineering)

Nanyang Technological University • Singapore

Presencial
SGD 90.000 - 135.000
Senior Assistant Manager, EdTech Enablement
Senior Assistant Manager, EdTech Enablement

Nanyang Technological University • Singapore

Híbrido
SGD 90.000 - 120.000
Work-From-Home options
Medical coverage
Employee engagement activities