Remote Data Annotation Specialist (Milan)

RemoExperts; Lombardia; Remote; EUR 37,000 - 61,000; Full time

Job summary

Rex.zone is seeking a Remote Data Annotation Specialist (Milan). You will label, review, and QA training data for NLP and computer vision tasks, following clear rubrics to improve model quality and safety.

This full-time remote role supports Milan-based availability, with competitive pay in USD per hour. You will perform RLHF ranking, evaluation, and documentation of annotation decisions across multiple time zones.

Qualifications

• 3+ years in data annotation or ML data operations.
• Strong written English and ability to follow detailed rubrics.
• Familiarity with LLM evaluation concepts (preference ranking, instruction tuning).
• Comfort with annotation tools and structured labeling schemas.
• Ability to maintain high accuracy under throughput targets.
• Security-minded approach to handling potentially sensitive content.

Responsibilities

• Label, review, and QA training data using rubrics and guidelines.
• Perform prompt evaluation, rubric scoring, and RLHF ranking.
• Execute QA evaluation using gold sets, peer review, audits, and disagreements.
• Apply named entity recognition and taxonomy-based labeling.
• Support content safety labeling and escalate ambiguous cases.
• Document annotation decisions and ensure guideline compliance.

Skills

Data labeling

English proficiency

Annotation tools

Quality assurance

RLHF concepts

Job description

Remote Data Annotation Specialist (Milan) — Remote (US-listed)

Rex.zone connects Milan-based talent with full-time remote AI data work supporting LLM training pipelines, RLHF, and model evaluation.

You will label, review, and QA training data using clear rubrics and annotation guidelines to improve model quality and safety.

What You Will Do

• Produce labeled examples for NLP and computer vision annotation tasks
• Perform prompt evaluation, rubric scoring, and RLHF preference ranking/response ranking
• Execute QA evaluation using gold sets, peer review, audits, and disagreement resolution
• Apply named entity recognition (span labeling) and taxonomy-based labeling
• Support content safety labeling (policy-driven categories) and escalata ambiguous cases
• Document annotation decisions and ensure annotation guidelines compliance

Core Workflows

• RLHF ranking and instruction-following grading
• Inter-annotator agreement checks and training data quality audits
• Adversarial/edge-case prompt evaluation
• Computer vision annotation (bounding boxes, polygons, keypoints)

Requirements

• 3+ years in data annotation, data labeling, QA evaluation, or ML data operations
• Strong written English and ability to follow detailed rubrics
• Familiarity with LLM evaluation concepts (preference ranking, instruction tuning)
• Comfort with annotation tools and structured labeling schemas
• Ability to maintain high accuracy under throughput targets
• Security-minded approach to handling potentially sensitive content

Employment Notes

This is a FULL_TIME Remote role aligned to Milan-based availability; projects may span multiple time zones depending on client needs.

Pay: Competitive hourly rate: $30–$50 per hour (USD)