Remote Data Annotation Specialist for AI Training & QA

Rex.zone; Denver (CO); Remote; USD 42,000 - 54,000; Full time

Job summary

Rex.zone is seeking remote data annotators to join our Denver-based program supporting AI systems and LLM training pipelines. You will label data for NLP tasks, perform RLHF-style ranking, evaluate prompts, and contribute to content safety labeling while following strict guidelines.

This is a full-time remote role with US-based candidates welcome; you will work with web-based annotation tools, adhere to confidential data handling, and help drive quality through rigorous self- and peer reviews.

Qualifications

• Mid-Senior experience in data annotation, labeling, QA review, or AI/ML evaluation workflows.
• Strong written communication for rationales and guideline interpretations.
• Ability to follow rubrics with high consistency and quality standards.
• Comfort with web-based annotation tools and task tracking.
• Understanding of LLM training concepts (supervised fine-tuning data, RLHF, evaluation sets).

Responsibilities

• Perform data labeling for NLP tasks including named entity recognition, sentiment, intent, and taxonomy-based classification.
• Complete RLHF-style ranking and preference labeling to support LLM evaluation and alignment.
• Run prompt evaluation and response grading for accuracy, helpfulness, policy compliance, and reasoning quality.
• Execute content safety labeling using client rubrics (hate/harassment, self-harm, sexual content, violence).
• Annotate computer vision data when needed (bounding boxes, polygons, keypoints, segmentation masks).
• Apply annotation guidelines compliance, flag ambiguous items, and document edge cases.
• Conduct QA evaluation (self-check and peer review), track error types, and drive quality improvements.
• Maintain secure handling of sensitive data and adhere to confidentiality and platform policies.

Job description

Rex.zone is seeking remote data annotators to join our Denver-based program supporting AI systems and LLM training pipelines. You will label data for NLP tasks, perform RLHF-style ranking, evaluate prompts, and contribute to content safety labeling while following strict guidelines.

This is a full-time remote role with US-based candidates welcome; you will work with web-based annotation tools, adhere to confidential data handling, and help drive quality through rigorous self- and peer reviews.