Remote Data Labeling & Evaluation Specialist

Rex.zone; Warszawa; Remote; PLN 160,000 - 267,000; Full time

Job summary

Rex.zone is hiring for full-time, fully remote data labeling and evaluation roles supporting AI/ML development. You will create and validate labeled datasets used in NLP, computer vision annotation, content safety labeling, and LLM training pipelines, with a strong emphasis on training data quality and annotation guidelines compliance.

Responsibilities include text and multimodal labeling, LLM evaluation, RLHF-style tasks, QA workflows, and documenting edge cases while collaborating with project

Qualifications

• Mid-senior experience in data annotation or evaluation roles.
• Strong attention to detail and ability to follow annotation guidelines.
• Familiarity with QA concepts like gold data and audits.

Responsibilities

• Execute text annotation tasks (classification, named entity recognition) and multimodal labeling (image/video/audio).
• Perform rubric-based large language model evaluation, including prompt evaluation and response grading.
• Complete RLHF-style preference tasks (pairwise ranking, rationale capture) aligned to evaluation rubrics.
• Run QA evaluation workflows (spot checks, gold set validation, sampling audits, disagreement analysis).
• Document edge cases, flag policy/privacy issues, and support content safety labeling escalations.
• Collaborate asynchronously with project leads to clarify guidelines and improve dataset consistency.

Skills

Data annotation

QA evaluation

Remote work

Job description

Rex.zone is hiring for full-time, fully remote data labeling and evaluation roles supporting AI/ML development. You will create and validate labeled datasets used in NLP, computer vision annotation, content safety labeling, and LLM training pipelines, with a strong emphasis on training data quality and annotation guidelines compliance.

Responsibilities include text and multimodal labeling, LLM evaluation, RLHF-style tasks, QA workflows, and documenting edge cases while collaborating with project