Remote Data Labeling Specialist (United States)

Rex.zone; United States; On-site; USD 55,000 - 78,000; Full time

Job summary

Rex.zone is seeking a Remote Data Labeling Specialist to label and evaluate multi-modal AI training data (text, images, and conversations) to improve model quality. You will follow rubrics, maintain guidelines compliance, and support production systems with high-precision annotations.

In this fully remote, full-time role within the United States, you will perform labeling for LLM pipelines, RLHF tasks, and content safety tagging, while documenting edge cases and proposing taxonomy updates based

Qualifications

• Experience with production data labeling or QA evaluation workflows.
• Ability to work with detailed taxonomies, rubrics, and policy language.
• Maintain throughput while protecting data quality.
• Clear asynchronous communication in a fully remote environment.

Responsibilities

• Label and annotate multimodal data for model training.
• Perform RLHF tasks including preference comparisons and rubric scoring.
• Evaluate LLM prompts for helpfulness, factuality, grounding, relevance, and instruction-following.
• Apply named entity recognition and span labeling per rubrics.
• Support computer vision annotation (boxes, polygons, keypoints) as needed.
• Run QA audits and inter-annotator agreement checks.
• Document edge cases and propose taxonomy updates with evidence.

Skills

Data labeling

QA workflows

Rubrics & taxonomies

Remote communication

Job description

Role Overview

As a Remote Data Labeling Specialist, you will label and evaluate multi-modal AI training data (text, images, and conversations) to improve model performance in production systems. You will follow detailed rubrics, maintain strong annotation guidelines compliance, and contribute to measurable training data quality improvements.

What You Will Do

• Execute data labeling and data annotation for LLM training pipelines (classification, extraction, ranking)
• Perform RLHF workflows including preference comparisons, rationale tagging, and rubric-based scoring
• Conduct LLM evaluation and prompt evaluation for helpfulness, factuality, grounding, relevance, and instruction-following
• Apply named entity recognition and span labeling with strict rubric adherence
• Support computer vision annotation (bounding boxes, polygons, keypoints, image categorization) when needed
• Run QA evaluation audits, inter-annotator agreement checks, and targeted reviews to ensure training data quality
• Perform content safety labeling and policy-based moderation tagging
• Document edge cases, propose taxonomy updates, and elevate ambiguous items with evidence

Required Qualifications

• Experience with production data labeling or related QA evaluation workflows
• Comfort working with detailed taxonomies, rubrics, and policy language
• Ability to maintain throughput while protecting training data quality
• Clear asynchronous communication in a fully remote environment

Remote Work Details

This is a full-time Remote role within the United States, with calibration sessions and QA checkpoints handled online. You must follow confidentiality and secure dataset handling requirements.