Remote AI Data Annotator — RLHF & QA Specialist

RemoExperts; Lombardia; Remote; EUR 37,000 - 61,000; Full time

Job summary

Rex.zone is seeking a Remote Data Annotation Specialist (Milan). You will label, review, and QA training data for NLP and computer vision tasks, following clear rubrics to improve model quality and safety.

This full-time remote role supports Milan-based availability, with competitive pay in USD per hour. You will perform RLHF ranking, evaluation, and documentation of annotation decisions across multiple time zones.

Qualifications

• 3+ years in data annotation or ML data operations.
• Strong written English and ability to follow detailed rubrics.
• Familiarity with LLM evaluation concepts (preference ranking, instruction tuning).
• Comfort with annotation tools and structured labeling schemas.
• Ability to maintain high accuracy under throughput targets.
• Security-minded approach to handling potentially sensitive content.

Responsibilities

• Label, review, and QA training data using rubrics and guidelines.
• Perform prompt evaluation, rubric scoring, and RLHF ranking.
• Execute QA evaluation using gold sets, peer review, audits, and disagreements.
• Apply named entity recognition and taxonomy-based labeling.
• Support content safety labeling and escalate ambiguous cases.
• Document annotation decisions and ensure guideline compliance.

Skills

Data labeling

English proficiency

Annotation tools

Quality assurance

RLHF concepts

Job description

Rex.zone is seeking a Remote Data Annotation Specialist (Milan). You will label, review, and QA training data for NLP and computer vision tasks, following clear rubrics to improve model quality and safety.

This full-time remote role supports Milan-based availability, with competitive pay in USD per hour. You will perform RLHF ranking, evaluation, and documentation of annotation decisions across multiple time zones.