AI Data Annotation and Evaluation Specialist (12-month contract)

Career International FOS Pte Ltd; Santo Niño 1st; On-site; PHP 391,000 - 614,000; Full time

Job summary

Career International FOS Pte Ltd is seeking an Annotation Specialist to support AI model evaluation. You will work across languages such as Thai, Korean, Japanese, French, German, Portuguese (Brazil), and Spanish, ensuring editorial quality and scalability.

You will conduct vendor annotation quality checks, help develop guidelines, design and refine system prompts, and analyze data for error trends, collaborating with project managers in fast-paced projects.

Qualifications

• Bachelor's degree with strong academic performance.
• Experience in writing and optimizing AI system prompts.
• Proficiency in English with excellent written and verbal communication.
• Fluency in Thai, Korean, Japanese, French, German, Portuguese (Brazil) or Spanish.
• Strong data sensitivity and analytical skills.
• Ability to balance individual judgment with teamwork in fast-paced environments.
• Self-motivation and ability to handle multiple priorities.

Responsibilities

• Execute annotation workflows and guidelines for AI model evaluation.
• Identify quality risks and error patterns; report to stakeholders.
• Conduct systematic quality checks on vendor annotations.
• Maintain detailed audit trails and provide actionable feedback.
• Develop annotation guidelines with project managers.
• Design, test, and refine system prompts for evaluation tasks.
• Create prompt libraries and best practices for consistency.
• Analyze annotation datasets to identify error trends.

Skills

AI prompt writing

Multilingual proficiency

English communication

Quality assurance

Analytical thinking

Education

Bachelor's degree

Job description

About the role

Involving on one of following languages: Thai, Korean, Japan, French, German, Portuguese (Brazil), Spanish

This role involves executing annotation workflows and guidelines for AI model evaluation, ensuring editorial quality, accuracy, and scalability. You will conduct systematic quality checks on vendor-delivered annotations, collaborate with project managers to develop annotation guidelines for pioneering AI projects, design and refine system prompts, and analyze annotation datasets to identify error trends and performance gaps.

Key responsibilities

• Execute annotation workflows and guidelines for AI model evaluation, ensuring editorial quality, accuracy, and scalability
• Proactively identify quality risks and error patterns, and communicate findings to stakeholders to drive continuous improvement
• Conduct systematic quality checks on vendor-delivered annotations to verify alignment with project expectations and training objectives
• Maintain detailed audit trails and provide clear and actionable feedback to enhance annotator performance
• Collaborate with project managers to develop annotation guidelines for pioneering AI projects
• Execute hands‑on annotation when workflows are not yet established, helping to shape scalable processes
• Design, test, and refine system prompts to improve model outputs for evaluation tasks
• Develop prompt libraries and best practices that enable efficient and consistent workflows across projects
• Analyze annotation datasets to identify error trends and performance gaps
• Stay current with advancements in large language models and AI evaluation methodologies

Requirements

• Bachelor's degree with strong academic performance
• Experienced in writing and optimizing AI system prompts
• Professional proficiency in English with excellent written and verbal communication skills
• Professional proficiency or native fluency in at least one of the following languages: Thai, Korean, Japan, French, German, Portuguese (Brazil), Spanish
• Strong data sensitivity and analytical skills to identify quality issues and error patterns
• Ability to apply critical thinking to balance individual judgment with collaborative teamwork in fast-paced environments
• Self‑motivated with a strong sense of responsibility, able to adapt to ambiguity and manage multiple priorities within tight cycles
• Genuine interest in AI, large language models, and human‑computer interaction
• Excellent cross‑team collaboration skills and willingness to continuously learn and explore new technologies