Remote Data Annotation Specialist for AI/ML Training

Rex.zone; California (MO); Remote; USD 42,000 - 66,000; Full time

Job summary

Rex.zone offers remote, full-time data annotation work supporting AI/ML and LLM training pipelines. You will create high-quality labeled datasets and human feedback signals, including RLHF ranking, prompt evaluation, and QA evaluation, while maintaining strict annotation guidelines compliance and training data quality.

You will adapt to asynchronous review cycles, document decisions thoroughly, and contribute to ongoing improvements in annotation quality and model performance.

Qualifications

• Experience with structured labeling workflows and web-based annotation tools.
• Strong written reasoning and ability to follow detailed guidelines with high precision.
• Comfort working asynchronously with reviewer feedback loops and QA processes.

Responsibilities

• Label and categorize text, image, and multimodal data using defined taxonomies and policies.
• Perform RLHF pairwise preference ranking and write clear rationales/justifications.
• Execute prompt evaluation and response quality scoring for LLM evaluation.
• Run QA evaluation audits, calibration tasks, and gold-set checks to improve consistency.
• Complete named entity recognition (NER) and span labeling for NLP datasets.
• Support content safety labeling and sensitive content handling per policy.
• Maintain throughput and accuracy targets with detailed decision logs for reviewer traceability.

Skills

Structured labeling workflows

Web-based annotation tools

Attention to detail

Strong written reasoning

Tools

Annotation tools

Job description

Rex.zone offers remote, full-time data annotation work supporting AI/ML and LLM training pipelines. You will create high-quality labeled datasets and human feedback signals, including RLHF ranking, prompt evaluation, and QA evaluation, while maintaining strict annotation guidelines compliance and training data quality.

You will adapt to asynchronous review cycles, document decisions thoroughly, and contribute to ongoing improvements in annotation quality and model performance.