AI Data Labeling Jobs Canada

Rex.zone; United States; Remote; USD 41,000 - 69,000; Full time

Benefits offered by this job

Remote work

Asynchronous collaboration

Job summary

Rex.zone is hiring for a remote, full-time AI data labeling role focused on producing and evaluating high-quality labeled training data for modern AI/ML systems, including LLM training pipelines, RLHF workflows, and computer vision annotation.

You will label and review datasets, perform QA, ranking, prompt evaluation, and NLP tagging while adhering to detailed rubrics. This role emphasizes precision, remote collaboration, and continuous quality improvement.

Qualifications

• Mid-Senior experience in data annotation, QA, or AI/ML production operations.
• Ability to follow detailed rubrics and deliver consistent labeling decisions at scale.
• Familiarity with LLM evaluation concepts and training data quality practices.
• Strong written communication for documenting QA findings.

Responsibilities

• Label and review text, image, and multimodal datasets using project rubrics and taxonomies.
• Perform QA evaluation, spot checks, calibration rounds, and adjudication to maintain training data quality.
• Complete RLHF-style preference ranking, rubric-based scoring, and prompt-response grading.
• Conduct prompt evaluation for helpfulness, correctness, and safety.
• Apply named entity recognition (NER) tagging and other NLP labeling tasks.
• Support content safety labeling and safety policy enforcement where required.
• Document edge cases and propose guideline updates to improve annotation guidelines compliance.
• Track error patterns and contribute to error analysis for model performance improvement.

Skills

Data annotation

QA evaluation

NLP labeling

Documentation

Job description

Overview

Rex.zone is hiring for a remote, full-time AI data labeling role focused on producing and evaluating high-quality labeled training data for modern AI/ML systems, including LLM training pipelines, RLHF workflows, and computer vision annotation.

What You Will Do

• Label and review text, image, and multimodal datasets using project rubrics and taxonomies
• Perform QA evaluation, spot checks, calibration rounds, and adjudication to maintain training data quality
• Complete RLHF-style preference ranking, rubric-based scoring, and prompt-response grading
• Conduct prompt evaluation for helpfulness, correctness, and safety
• Apply named entity recognition (NER) tagging and other NLP labeling tasks
• Support content safety labeling and safety policy enforcement where required
• Document edge cases and propose guideline updates to improve annotation guidelines compliance
• Track error patterns and contribute to error analysis for model performance improvement

Required Qualifications

• Mid-Senior experience in data annotation, QA, evaluation, or AI/ML production operations
• Ability to follow detailed rubrics and deliver consistent labeling decisions at scale
• Familiarity with LLM evaluation concepts and training data quality practices
• Strong written communication for documenting decisions and QA findings

Work Arrangement

Remote, full-time. Work is delivered through Rex.zone project systems with asynchronous collaboration and periodic calibration.

Compensation

$30–$50 per hour (base pay; depends on project scope and complexity).