Polish AI Annotation & Evaluation Specialist

The lemon Consulting; Lisboa; On-site; EUR 19,000 - 26,000; Full time

Benefits offered by this job

Modern office in city center

Full-time in Lisbon

Meal allowance

Training and guidance

Coaching and mentoring

Health and wellbeing initiatives

International environment

Job summary

The lemon Consulting in Lisbon, Portugal is seeking a Customer Support Representative to review and annotate AI-generated responses, ensuring quality, accuracy and policy compliance in a fast-paced environment. You will work on-site in central Lisbon, use AI annotation platforms, and collaborate with multilingual teams.

Requirements include native Polish, strong English, analytical mindset and careful interpretation of guidelines.

Qualifications

• Native-level proficiency in Polish and excellent written communication
• Good command of English at least B2
• Strong understanding of Generative AI, LLMs, and AI annotation/evaluation platforms
• Experience in AI data annotation, LLM evaluation, RLHF, SFT, Quality Assurance or related analytical environment is desirable
• Excellent analytical and critical thinking with high attention to detail
• Ability to interpret complex guidelines and apply policies consistently

Responsibilities

• Review and evaluate AI-generated responses using detailed quality and annotation guidelines
• Label and annotate language model outputs accurately and consistently
• Assess AI content for quality, relevance, factual accuracy, safety and policy compliance
• Apply complex annotation guidelines across different scenarios and content types
• Identify inconsistencies and escalate potential quality issues
• Review and analyse structured text, language data and AI-generated content
• Maintain high accuracy and quality while meeting productivity targets
• Provide accurate evaluations and actionable feedback to support model improvement
• Maintain confidentiality and follow data security and quality standards
• Adapt to updated guidelines and project requirements

Skills

Polish (native)

English (B2)

AI annotation platforms

AI evaluation

Job description

The lemon Consulting in Lisbon, Portugal is seeking a Customer Support Representative to review and annotate AI-generated responses, ensuring quality, accuracy and policy compliance in a fast-paced environment. You will work on-site in central Lisbon, use AI annotation platforms, and collaborate with multilingual teams.

Requirements include native Polish, strong English, analytical mindset and careful interpretation of guidelines.