Multilingual LLM Data Evaluator (Bahasa Melayu and English)

Chemin; Kuala Lumpur; Remote; MYR 100,000 - 150,000; Full time

Job summary

A forward-thinking AI enablement company in Malaysia is seeking LLM Evaluators to validate AI-generated outputs. Candidates must be Malaysian citizens fluent in Bahasa Malaysia and English, with excellent attention to detail. This 100% remote role requires working 5-8 hours per day, ensuring high standards in evaluating AI language models. Gain valuable experience in shaping the future of language technology from the comfort of your home.

Benefits

Flexible work hours

Remote work opportunity

Gain experience in AI solutions

Qualifications

• Malaysian citizen fluent in English and Bahasa Melayu (reading and writing).
• Able to evaluate logic, language clarity, and reasoning structure.
• Capable of detecting unnatural language and adapting it while maintaining meaning.

Responsibilities

• Deliver precise validations of response outputs with minimal revisions.
• Evaluate LLM prompts for accuracy and conversational alignment.
• Maintain the original meaning while improving grammar and clarity.

Skills

Fluency in Bahasa Malaysia

Fluency in English

Attention to detail

Judgment

Time management

Job description

Chemin is partnering with a well-established enterprise with an innovation arm focused on emerging technologies, infrastructure, and AI-related initiatives. For this project, we are seeking individuals as LLM Evaluators who can read and write Bahasa Malaysia (BM) and English fluently.

You will evaluate AI-generated outputs by reviewing the response accuracy and reasoning or logic, using the chat history strictly as context. This role requires strong language skills, judgment, attention to detail, and the ability to identify logical, grammatical, and structural issues without altering the original meaning of outputs widely.

The key to your success is to ensure a high standard of excellence in performing validations, as quality is critical to enhancing AI models. Ultimately, alongside the Delivery Team at Chemin, you will shape how Malaysian languages are represented in the next generation of powerful large language models by refining a high volume and quality dataset.

• Deliver precise validations of response outputs with minimal to no revisions.
• Evaluate LLM prompts:
  • Ensure the response is accurate and aligned with the conversational context.
  • Verify that the reasoning supports the response logic.
  • Rewrite responses or reasoning where required, depending on the severity of issues.
• Adapt text to sound natural in the service languages (Bahasa Malaysia and English), especially in cases where LLM outputs are affected by translation-related grammar errors.
• Maintain the original meaning of the response output while improving sentence structure, grammar, and clarity for minor errors.
• Discard data points with entirely illogical response outputs or reasoning, as these are unusable for AI training.
• Proactively communicate challenges, queries, or concerns with Project Managers to ensure smooth operations.
• Meet key deadlines while maintaining quality and consistency in work output.

Qualifications

• Malaysian citizen fluent in English and Bahasa Melayu (reading and writing).
• Able to evaluate logic, language clarity, and reasoning structure.
• Capable of detecting unnatural language caused by machine translation and adapt it while maintaining meaning.
• Attention to detail and a high level of accuracy in detecting errors in spelling, grammar, and conversation flow.
• Efficient time management skills to meet and exceed daily deliverables.
• Professional attitude: responsive and inquisitive to ask clarifying questions.
• Possess a laptop or desktop computer with a stable internet connection.
• High attention to detail and ability to follow strict linguistic and structural guidelines.
• Comfortable working with LLM-generated content and chat-style datasets.
• Bonus: prior experience in LLM evaluation or validation and data annotation projects, especially in linguistics.

Work Arrangements

• Available for at least 5-8 hours per day, including on weekends when necessary. Flexible core work hours from 9:00 am - 6:00 pm.
• Able to commit to the project from November 19th to November 30th, 2025.

• 100% remote work opportunity.
• Work from the comfort of your own home.
• Be part of a forward-thinking AI enablement company in Malaysia.
• Gain invaluable work experience on an AI solution that represents Malaysian languages and dialects.

Accelerate AI development with Chemin by training next-generation language models!