Remote AI Data Labeling Specialist

Rex.zone; San Diego (CA); On-site; USD 42,000 - 66,000; Full time

Job summary

Rex.zone is seeking a Remote Data Annotation Specialist to create and evaluate labeled datasets that train and validate modern AI/ML systems for US-based projects. You will work across NLP labeling, computer vision annotation, RLHF preference ranking, and prompt-response evaluation while following versioned guidelines and meeting clear quality targets.

This is a Remote, FULL_TIME role with asynchronous collaboration and QA review sessions. Location reference: San Diego, CA, US.

Qualifications

• Mid-Senior experience in data annotation, data labeling, QA evaluation, or LLM evaluation in production or vendor environments.
• Strong reading comprehension and attention to detail for rubric-based judgment and policy-based labeling.
• Comfort with annotation tools, task queues, and strict annotation guidelines compliance.
• Ability to explain decisions with concise written rationales.
• Familiarity with NLP concepts such as named entity recognition and taxonomy-driven labeling (preferred).

Responsibilities

• Execute data labeling tasks across NLP and computer vision annotation projects (text span labeling, classification, bounding boxes, segmentation, attribute tagging).
• Perform RLHF and large language model evaluation by ranking outputs, scoring prompt-response pairs, and identifying instruction-following failures.
• Run QA evaluation workflows including self-checks, peer reviews, and gold-standard audits to improve training data quality and consistency.
• Apply content safety labeling taxonomies and elevate uncertain cases using defined escalation paths.
• Write clear rationales, document edge cases, and propose guideline updates when recurring ambiguities appear.
• Track throughput and quality metrics (accuracy, agreement rates, defect density) to support model performance improvement.

Skills

Data annotation

QA evaluation

Documentation

Attention to detail

Written rationale

Job description

Rex.zone is seeking a Remote Data Annotation Specialist to create and evaluate labeled datasets that train and validate modern AI/ML systems for US-based projects. You will work across NLP labeling, computer vision annotation, RLHF preference ranking, and prompt-response evaluation while following versioned guidelines and meeting clear quality targets.

This is a Remote, FULL_TIME role with asynchronous collaboration and QA review sessions. Location reference: San Diego, CA, US.