AI Training Data Labeling Specialist (Remote)

Rex.zone; Austin (TX); On-site; USD 41,328 - 68,880; Full time

Job summary

Rex.zone is hiring a Remote Data Labeling Specialist to produce high-quality training data for AI/ML systems. You will complete data labeling, RLHF preference ranking, QA evaluation audits, prompt evaluation, and content safety labeling to support LLM training pipelines and computer vision workflows.

The role involves labeling text, images, and multimodal content using web-based tools, performing RLHF and QA tasks, and collaborating with cross-functional teams to ensure data quality and

Qualifications

• Mid-Senior experience delivering production data labeling / data annotation.
• Strong annotation guideline compliance and consistency across large volumes.
• Experience with QA evaluation workflows (error taxonomy, root-cause analysis).
• Familiarity with RLHF and LLM evaluation rubrics.
• Clear written communication for rationale and edge-case documentation.
• Comfort with web labeling interfaces and review queues.

Responsibilities

• Label and categorize text, images, and multimodal content using web-based annotation tools and strict guidelines.
• Perform RLHF work: preference ranking, pairwise comparisons, rubric-based scoring, and rationale documentation.
• Execute QA evaluation (sampling, audits, defect tracking) and provide feedback to improve training data quality.
• Run prompt evaluation for helpfulness, harmlessness, and factuality within defined policies.
• Apply named entity recognition (NER) and span labeling for structured extraction tasks.
• Support computer vision annotation (bounding boxes, polygons, segmentation, keypoints) as needed.
• Document edge cases and propose guideline clarifications to improve reviewer agreement.
• Collaborate with operations and engineering stakeholders to improve throughput without sacrificing quality.

Skills

Data labeling

Quality assurance

RLHF knowledge

LLM evaluation rubrics

Clear written communication

Web labeling interfaces

Tools

Annotation tools

Job description

Rex.zone is hiring a Remote Data Labeling Specialist to produce high-quality training data for AI/ML systems. You will complete data labeling, RLHF preference ranking, QA evaluation audits, prompt evaluation, and content safety labeling to support LLM training pipelines and computer vision workflows.

The role involves labeling text, images, and multimodal content using web-based tools, performing RLHF and QA tasks, and collaborating with cross-functional teams to ensure data quality and