Remote Data Labeling Jobs Toronto

Rex.zone; United States; Remote; USD 41,000 - 69,000; Full time

Job summary

Rex.zone seeks a remote data labeling specialist to deliver accurate labels across text, image, and multimodal data, following strict rubric-based guidelines. You will document edge cases and help improve annotation guidelines while working fully remotely with Rex.zone project tools.

You will perform RLHF preference ranking with rationale writing for LLM training pipelines, execute prompt evaluations, and participate in QA workflows like calibration and audits.

Qualifications

• Experience in data labeling, data annotation, or QA evaluation for AI/ML datasets.
• Strong written communication for clear rationales and edge-case documentation.
• Comfort applying rubrics and maintaining annotation guideline compliance.
• Understanding of training data quality concepts (agreement, consistency, bias).
• Ability to work independently in a remote setting with dependable execution.

Responsibilities

• Label text, image, and multimodal data following strict annotation guidelines compliance
• Perform RLHF preference ranking, including rationale writing for LLM training pipelines
• Execute prompt evaluation against rubrics (helpfulness, harmlessness, factuality)
• Run QA evaluation workflows: calibration, consensus checks, sampling audits, and adjudication notes
• Complete NLP labeling such as named entity recognition and text classification
• Support computer vision annotation (bounding boxes, polygons, keypoints, segmentation)
• Apply content safety labeling for policy categories and risk severity
• Flag ambiguous cases and propose guideline updates to reduce label noise

Skills

Data labeling

QA evaluation

Written communication

Remote work

Job description

About Rex.zone

Rex.zone supports AI labs, tech startups, BPOs, and annotation vendors with scalable human feedback and training data operations. This role focuses on high-quality remote data labeling to improve large language model evaluation and model performance.

About Rex.zone

Rex.zone supports AI labs, tech startups, BPOs, and annotation vendors with scalable human feedback and training data operations. This role focuses on high-quality remote data labeling to improve large language model evaluation and model performance.

About The Role

You will deliver accurate labels and written rationales using rubric-based evaluation, document edge cases, and help improve annotation guidelines. Work is performed fully remote using Rex.zone workflows and project tools.

What You Will Do

• Label text, image, and multimodal data following strict annotation guidelines compliance


• Perform RLHF preference ranking, including rationale writing for LLM training pipelines


• Execute prompt evaluation against rubrics (helpfulness, harmlessness, factuality)


• Run QA evaluation workflows: calibration, consensus checks, sampling audits, and adjudication notes


• Complete NLP labeling such as named entity recognition and text classification


• Support computer vision annotation (bounding boxes, polygons, keypoints, segmentation)


• Apply content safety labeling for policy categories and risk severity


• Flag ambiguous cases and propose guideline updates to reduce label noise

Required Qualifications

• Experience in data labeling, data annotation, or QA evaluation for AI/ML datasets


• Strong written communication for clear rationales and edge-case documentation


• Comfort applying rubrics and maintaining annotation guideline compliance


• Understanding of training data quality concepts (agreement, consistency, bias)


• Ability to work independently in a remote setting with dependable execution

Quality & Success Metrics

• Guideline compliance and QA pass rate


• Calibration agreement and audit outcomes


• Turnaround time balanced with accuracy
  
• Measurable improvements from cleaner labels and better preference data

Compensation

Competitive hourly base pay: $30–$50 per hour (USD), based on scope, label complexity, and QA responsibility.