Arabic AI Data Evaluator for LLMs — Lisbon

CloserHire; Lisboa; On-site; EUR 12,000 - 14,000; Full time

Benefits offered by this job

Onboarding program

Private health insurance

Meal allowance

International environment

Career development opportunities

Job summary

CloserHire is recruiting an Arabic-speaking AI Data Evaluation Specialist for a Lisbon-based project on advanced AI systems. You will review and improve AI-generated responses, ensure language accuracy, safety, and policy compliance, on a 24/7 rotating schedule with two days off. Relocation support is included.

Ideal candidates are native Arabic speakers with strong English and a solid understanding of AI evaluation methodologies. This role offers a dynamic, international environment in Lisbon.

Qualifications

• Native Arabic speaker with high-level English proficiency
• Experience with AI data annotation or LLM evaluation is preferred
• Ability to understand and apply complex evaluation guidelines

Responsibilities

• Review and evaluate AI-generated responses using detailed annotation guidelines
• Label and annotate model outputs for accuracy and consistency
• Assess content for quality, relevance, factual accuracy, safety and policy compliance
• Apply complex annotation guidelines across diverse content scenarios
• Identify inconsistencies and edge cases, escalate when needed
• Analyze structured text and datasets to support model improvement
• Deliver high-quality work meeting productivity, quality, and accuracy targets
• Maintain confidentiality and comply with project standards and security requirements
• Contribute to ongoing training and improvement of AI systems through evaluations and feedback (24/7 rotating schedule)

Skills

Arabic fluency

English B2+

Generative AI / LLMs

AI data annotation

Attention to detail

Guideline interpretation

Job description

CloserHire is recruiting an Arabic-speaking AI Data Evaluation Specialist for a Lisbon-based project on advanced AI systems. You will review and improve AI-generated responses, ensure language accuracy, safety, and policy compliance, on a 24/7 rotating schedule with two days off. Relocation support is included.

Ideal candidates are native Arabic speakers with strong English and a solid understanding of AI evaluation methodologies. This role offers a dynamic, international environment in Lisbon.