Senior Data Annotator - Remote AI Training & Evaluation

Rex.zone; Seattle (WA); On-site; USD 41,328 - 68,880; Full time

Job summary

Rex.zone is hiring a Senior Data Annotator to produce and review high-quality training data for AI systems across NLP, computer vision, and multimodal evaluation. This is a Remote, full-time role focused on annotation guidelines compliance, content safety labeling, and evaluation workflows that directly improve data quality and model reliability.

The role involves executing complex labeling tasks across text, image, and multimodal data, performing RLHF ranking, and contributing to QA workflows.

Qualifications

• Mid-Senior experience in data annotation, labeling, or QA evaluation.
• High accuracy and consistency following (and refining) annotation guidelines.
• Familiarity with RLHF concepts, prompt evaluation, and LLM training/evaluation workflows.
• Experience with named entity recognition or similar structured labeling tasks.
• Comfort with computer vision annotation and/or multimodal labeling.
• Strong written judgment for content safety labeling and policy-driven decisions.

Responsibilities

• Execute complex data labeling and review tasks across text, image, and multimodal datasets (NLP + computer vision annotation)
• Perform RLHF preference ranking, prompt evaluation, and LLM evaluation judgments to improve model behavior
• Run QA evaluation workflows including audits, calibration, adjudication, error analysis, and rubric enforcement
• Apply named entity recognition and taxonomy-based labeling for structured datasets
• Document edge cases, decisions, and guideline clarifications to strengthen annotation guidelines compliance
• Identify systemic label noise and recommend remediation to support model performance improvement

Skills

Data annotation

QA evaluation

RLHF concepts

LLM evaluation

Named entity recognition

Content safety labeling

Multimodal labeling

Written judgment

Job description

Rex.zone is hiring a Senior Data Annotator to produce and review high-quality training data for AI systems across NLP, computer vision, and multimodal evaluation. This is a Remote, full-time role focused on annotation guidelines compliance, content safety labeling, and evaluation workflows that directly improve data quality and model reliability.

The role involves executing complex labeling tasks across text, image, and multimodal data, performing RLHF ranking, and contributing to QA workflows.