Remote Data Annotation Jobs Amsterdam

Rex.zone; United States; Remote; USD 41,000 - 69,000; Full time

Job summary

Rex.zone is seeking a mid-senior Data Annotation professional for full-time remote work supporting distributed teams and rubric-driven annotation operations.

You will deliver high-precision labels for NLP, CV, and content safety tasks, participate in RLHF workflows, and document edge cases to improve inter-annotator agreement. Strong remote collaboration and clear writing required.

Qualifications

• 3+ years in data annotation, labeling, QA evaluation, or model evaluation for AI/ML.
• Experience with annotation guidelines compliance and quality calibration methods.
• Familiarity with RLHF, prompt evaluation, or LLM evaluation concepts.
• Strong analytical writing for issue reports and edge-case documentation.
• Comfort working remotely with distributed teams and structured throughput/quality metrics.

Responsibilities

• Annotate and review datasets for NLP, CV, and content safety tasks while following detailed rubrics
• Perform RLHF workflows (preference ranking, pairwise comparisons, justification capture)
• Execute QA evaluation: sampling plans, discrepancy analysis, and rework loops to reduce label noise
• Document edge cases and propose guideline updates to improve inter-annotator agreement
• Support prompt evaluation and model evaluation sets measuring helpfulness, correctness, and safety
• Maintain secure handling of sensitive content and adhere to confidentiality requirements

Skills

Data annotation

Quality calibration

RLHF concepts

Analytical writing

Remote collaboration

Job description

Remote Data Annotation Jobs Amsterdam (Remote)
Rex.zone connects experienced annotators with real-world AI/ML training workflows, including data labeling, RLHF evaluation, prompt evaluation, and training data quality improvements for large language models. This is a full-time remote role supporting distributed teams and rubric-driven annotation operations.

About The Role

As a mid-senior Data Annotation professional, you will deliver high-precision labels and human judgments used to train, fine-tune, and evaluate AI systems. Work spans NLP classification, named entity recognition, prompt evaluation for LLMs, RLHF preference ranking, computer vision annotation, and content safety labeling.

Key Responsibilities

• Annotate and review datasets for NLP, CV, and content safety tasks while following detailed rubrics
• Perform RLHF workflows (preference ranking, pairwise comparisons, justification capture)
• Execute QA evaluation: sampling plans, discrepancy analysis, and rework loops to reduce label noise
• Document edge cases and propose guideline updates to improve inter-annotator agreement
• Support prompt evaluation and model evaluation sets measuring helpfulness, correctness, and safety
• Maintain secure handling of sensitive content and adhere to confidentiality requirements

Required Qualifications

• 3+ years in data annotation, data labeling, QA evaluation, or model evaluation for AI/ML
• Experience with annotation guidelines compliance and quality calibration methods
• Familiarity with RLHF, prompt evaluation, or LLM evaluation concepts
• Strong analytical writing for issue reports and edge-case documentation
• Comfort working remotely with distributed teams and structured throughput/quality metrics

Compensation

Hourly rate: $30–$50/hr (USD), depending on scope and evaluation responsibilities.