Remote AI Data Labeling Specialist – NLP & RLHF

Rex.zone; Phoenix (AZ); Remote; USD 41,000 - 69,000; Full time

Job summary

Rex.zone is hiring Phoenix-based candidates for full-time remote roles focused on producing labeled training data and evaluation signals used in modern LLM training pipelines. You will label and review datasets across NLP, content safety, and computer vision, while supporting model performance improvement through consistent, high-quality evaluation.

The position involves executing data labeling for NLP tasks, RLHF preference work, prompt evaluation, and QA evaluation.

Qualifications

• Mid-to-senior experience in AI data annotation or QA evaluation workflows.
• Familiarity with RLHF and large language model evaluation concepts.
• Strong attention to detail for policy-based content safety labeling.

Responsibilities

• Execute data labeling for NLP tasks (text classification, summarization, NER).
• Perform RLHF preference work including pairwise ranking and rationale capture.
• Conduct prompt evaluation and response grading for instruction-following models.
• Complete QA evaluation with self-checks, peer review, and audits.
• Follow annotation guidelines, document edge cases, and flag ambiguous items.
• Meet throughput and accuracy targets with consistent rubric interpretation.

Skills

AI data annotation

Data labeling

QA evaluation

RLHF familiarity

Content safety labeling

Annotation tools

Job description

Rex.zone is hiring Phoenix-based candidates for full-time remote roles focused on producing labeled training data and evaluation signals used in modern LLM training pipelines. You will label and review datasets across NLP, content safety, and computer vision, while supporting model performance improvement through consistent, high-quality evaluation.

The position involves executing data labeling for NLP tasks, RLHF preference work, prompt evaluation, and QA evaluation.