Remote AI Data Labeling & QA Specialist

Rex.zone; United States; On-site; USD 41,000 - 69,000; Full time

Job description

Overview

Rex.zone is hiring for a remote, full-time AI data labeling role focused on producing and evaluating high-quality labeled training data for modern AI/ML systems, including LLM training pipelines, RLHF workflows, and computer vision annotation.

What You Will Do

• Label and review text, image, and multimodal datasets using project rubrics and taxonomies
• Perform QA evaluation, spot checks, calibration rounds, and adjudication to maintain training data quality
• Complete RLHF-style preference ranking, rubric-based scoring, and prompt-response grading
• Conduct prompt evaluation for helpfulness, correctness, and safety
• Apply named entity recognition (NER) tagging and other NLP labeling tasks
• Support content safety labeling and safety policy enforcement where required
• Document edge cases and propose guideline updates to improve annotation guidelines compliance
• Track error patterns and contribute to error analysis for model performance improvement

Required Qualifications

• Mid-Senior experience in data annotation, QA, evaluation, or AI/ML production operations
• Ability to follow detailed rubrics and deliver consistent labeling decisions at scale
• Familiarity with LLM evaluation concepts and training data quality practices
• Strong written communication for documenting decisions and QA findings

Work Arrangement

Remote, full-time. Work is delivered through Rex.zone project systems with asynchronous collaboration and periodic calibration.

Compensation

$30–$50 per hour (base pay; depends on project scope and complexity).