Remote AI Data Annotation Jobs in Phoenix

Rex.zone; Phoenix (AZ); Remote; USD 41,000 - 69,000; Full time

Job summary

Rex.zone is hiring Phoenix-based candidates for full-time remote roles focused on producing labeled training data and evaluation signals used in modern LLM training pipelines. You will label and review datasets across NLP, content safety, and computer vision, while supporting model performance improvement through consistent, high-quality evaluation.

The position involves executing data labeling for NLP tasks, RLHF preference work, prompt evaluation, and QA evaluation.

Qualifications

• Mid-to-senior experience in AI data annotation or QA evaluation workflows.
• Familiarity with RLHF and large language model evaluation concepts.
• Strong attention to detail for policy-based content safety labeling.

Responsibilities

• Execute data labeling for NLP tasks (text classification, summarization, NER).
• Perform RLHF preference work including pairwise ranking and rationale capture.
• Conduct prompt evaluation and response grading for instruction-following models.
• Complete QA evaluation with self-checks, peer review, and audits.
• Follow annotation guidelines, document edge cases, and flag ambiguous items.
• Meet throughput and accuracy targets with consistent rubric interpretation.

Skills

AI data annotation

Data labeling

QA evaluation

RLHF familiarity

Content safety labeling

Annotation tools

Job description

Remote AI Data Annotation Jobs in Phoenix (Remote)

Rex.zone is hiring Phoenix-based candidates for full-time remote roles focused on producing labeled training data and evaluation signals used in modern LLM training pipelines. You will label and review datasets across NLP, content safety, and (as needed) computer vision, while supporting model performance improvement through consistent, high-quality evaluation.

Key Responsibilities

• Execute data labeling for NLP tasks (text classification, summarization, and named entity recognition)
• Perform RLHF preference work including pairwise ranking, preference labeling, and rationale capture
• Conduct prompt evaluation and response grading for instruction-following models
• Complete QA evaluation (self-checks, peer review, audits) to maintain training data quality
• Follow annotation guidelines compliance, document edge cases, and elevate ambiguous items
• Meet throughput and accuracy targets while maintaining consistent rubric interpretation

Required Qualifications

• Mid-to-senior experience in AI data annotation, data labeling, or QA evaluation workflows
• Familiarity with RLHF and large language model evaluation concepts (or strong QA skills with ability to ramp)
• Strong attention to detail for policy-based content safety labeling
• Comfort with annotation tools, guideline documents, calibration sessions, and audit workflows

Workflow & Quality Metrics

• Quality measured via guideline adherence, audit scores, defect-rate tracking, and consensus alignment for RLHF rankings
• Structured queues with clear acceptance criteria, spot checks, and calibration to improve consistency

Location & Remote Notes

This is a Remote role for candidates based in Phoenix, US, collaborating asynchronously with distributed teams and following secure data handling practices.

Compensation

Hourly base pay range: $30-$50/hr (USD), based on scope, complexity, and demonstrated evaluation accuracy.