Lisbon AI Data Evaluation Specialist — Portuguese LLM QA

CloserHire; Lisboa; On-site; EUR 11,000 - 15,000; Full time

Benefits offered by this job

Office in Lisbon

Onboarding and training

Work on advanced AI project

International environment

Career development

Private health insurance

Meal allowance

Continuous learning

Modern office

Shape the future of AI

Job summary

CloserHire is recruiting a Portuguese-speaking AI Data Evaluation Specialist to join our Lisbon team, focusing on high-quality evaluation of AI-generated responses within next-generation LLM projects.

You will annotate, label, and assess content for accuracy, safety, and policy compliance, applying complex guidelines to diverse scenarios on a 24/7 rotating schedule. This is an office-based role in Lisbon.

Qualifications

• Native Portuguese fluency and strong English proficiency
• Experience in AI data annotation and LLM evaluation
• Familiarity with RLHF, supervised fine-tuning, QA and content labeling
• Ability to apply complex guidelines precisely
• Strong analytical thinking and attention to detail

Responsibilities

• Review and evaluate AI-generated responses using detailed guidelines
• Label and annotate outputs for accuracy and consistency
• Assess content for quality, safety, and policy compliance
• Apply guidelines across diverse scenarios
• Identify inconsistencies and escalate issues
• Analyze text, datasets and AI content to support model improvement
• Deliver high-quality work meeting targets
• Maintain confidentiality and security procedures
• Contribute to AI training and improvement through evaluations and feedback

Skills

Portuguese fluency

English proficiency

Generative AI knowledge

LLM evaluation

RLHF familiarity

SFT knowledge

Quality Assurance

Content labeling

Analytical thinking

Guidelines interpretation

Job description

CloserHire is recruiting a Portuguese-speaking AI Data Evaluation Specialist to join our Lisbon team, focusing on high-quality evaluation of AI-generated responses within next-generation LLM projects.

You will annotate, label, and assess content for accuracy, safety, and policy compliance, applying complex guidelines to diverse scenarios on a 24/7 rotating schedule. This is an office-based role in Lisbon.