AI Trainer – Annotation Quality & Performance

Thoth AI; Kuala Lumpur; On-site; MYR 60,000 - 80,000; Full time

Job summary

A technology company is seeking an AI Trainer to lead the training of annotators on AI data labeling. The role requires a Bachelor's in a relevant field and 2-4 years of experience in training or quality assurance. Ideal candidates will have strong communication and coaching skills, and experience with data accuracy metrics. This position is essential for ensuring high performance and operational scalability in AI projects.

Qualifications

• 2–4 years of experience in training, quality assurance, or performance management within BPO, AI annotation, or data operations.
• Proven ability to analyze performance data and implement structured training interventions.
• Knowledge of quality assurance frameworks and process improvement methodologies preferred.

Responsibilities

• Conduct onboarding sessions for new annotators to ensure full understanding of tools and workflows.
• Design and deliver training materials aligned with project requirements.
• Develop assessment tests to measure annotator readiness before production.

Skills

Presentation skills

Communication skills

Coaching skills

Analytical skills

Adaptability

Education

Bachelor’s degree in STEM, Education, Data Science, Psychology, or Linguistics

Tools

Airtable

Smartsheet

Jira

Job description

AI Trainer – Annotation Quality & Performance

We are seeking an experienced and detail-oriented AI Trainer to lead the training and development of annotators working on AI data labeling projects across text, audio, and video domains. This role is critical in ensuring that all annotators fully understand project guidelines, annotation rubrics, and quality standards — and are consistently able to deliver high-accuracy outputs that support Large Language Model (LLM) and AI system training.

Key Responsibilities

• Conduct onboarding sessions for new annotators, ensuring full understanding of annotation tools, workflows, and rubrics.
• Design, update, and deliver training materials, SOPs, and reference guides aligned with project requirements.
• Lead refresher training and knowledge calibration sessions to address performance gaps or rubric updates.
• Develop assessment tests and evaluation forms to measure annotator readiness before production deployment.
• Regularly review annotation performance reports with QA and PM teams to identify training needs.
• Coach underperforming annotators through one-on-one sessions and targeted improvement plans.
• Track learning progress, document training outcomes, and maintain training compliance records.
• Collaborate with QA to analyze common error patterns and translate them into training interventions.
• Participate in guideline calibration and contribute feedback to enhance annotation clarity and usability.
• Drive continuous improvement initiatives through training innovation and visual learning aids.
• Serve as the communication bridge between Project Management, Quality Assurance, and Operations.
• Support new project rollouts by preparing and certifying trainer-led readiness programs.

Qualifications

Required:

• Bachelor’s degree in STEM, Education, Data Science, Psychology, or Linguistics, or equivalent experience.
• 2–4 years of experience in training, quality assurance, or performance management within BPO, AI annotation, or data operations.
• Strong presentation, communication, and coaching skills.
• Proven ability to analyze performance data and implement structured training interventions.
• Familiarity with productivity monitoring tools and project management platforms (Airtable, Smartsheet, Jira, etc.).

Preferred:

• Experience working in AI data labeling, machine learning operations, or LLM annotation projects.
• Knowledge of quality assurance frameworks and process improvement methodologies.
• Certification in Six Sigma Green Belt, Kaizen, or Train-the-Trainer (TTT) is an advantage.
• Familiarity with data accuracy metrics and QA rubrics in AI or linguistic projects.

Core Competencies

• Instructional Excellence: Skilled in designing and delivering effective, learner-focused training sessions.
• Quality Orientation: Deep commitment to data precision and process standardization.
• Analytical Capability: Uses performance metrics to identify root causes and define improvement actions.
• Communication & Coaching: Inspires and guides annotators to consistently achieve high performance.
• Continuous Improvement: Applies process excellence methods (Six Sigma / Kaizen) to elevate training effectiveness.
• Adaptability: Thrives in dynamic, fast-changing AI project environments.

Purpose of the Role

The AI Trainer ensures that every annotator possesses the knowledge, discipline, and skill required to deliver data that meets project quality standards. By bridging operations, quality, and learning, this role is fundamental to achieving annotation consistency, operational scalability, and AI training excellence.