Remote Data Labeling Specialist for AI & LLM QA

Rex.zone; United States; Remote; USD 41,000 - 69,000; Full time

Job summary

Rex.zone seeks a remote data labeling specialist to deliver accurate labels across text, image, and multimodal data, following strict rubric-based guidelines. You will document edge cases and help improve annotation guidelines while working fully remotely with Rex.zone project tools.

You will perform RLHF preference ranking with rationale writing for LLM training pipelines, execute prompt evaluations, and participate in QA workflows like calibration and audits.

Qualifications

• Experience in data labeling, data annotation, or QA evaluation for AI/ML datasets.
• Strong written communication for clear rationales and edge-case documentation.
• Comfort applying rubrics and maintaining annotation guideline compliance.
• Understanding of training data quality concepts (agreement, consistency, bias).
• Ability to work independently in a remote setting with dependable execution.

Responsibilities

• Label text, image, and multimodal data following strict annotation guidelines compliance
• Perform RLHF preference ranking, including rationale writing for LLM training pipelines
• Execute prompt evaluation against rubrics (helpfulness, harmlessness, factuality)
• Run QA evaluation workflows: calibration, consensus checks, sampling audits, and adjudication notes
• Complete NLP labeling such as named entity recognition and text classification
• Support computer vision annotation (bounding boxes, polygons, keypoints, segmentation)
• Apply content safety labeling for policy categories and risk severity
• Flag ambiguous cases and propose guideline updates to reduce label noise

Skills

Data labeling

QA evaluation

Written communication

Remote work

Job description

Rex.zone seeks a remote data labeling specialist to deliver accurate labels across text, image, and multimodal data, following strict rubric-based guidelines. You will document edge cases and help improve annotation guidelines while working fully remotely with Rex.zone project tools.

You will perform RLHF preference ranking with rationale writing for LLM training pipelines, execute prompt evaluations, and participate in QA workflows like calibration and audits.