Remote AI Data Labeling & Evaluation Specialist

Rex.zone; United States; On-site; USD 42,000 - 68,000; Full time

Job summary

Rex.zone seeks a remote generalist to support structured AI/ML data operations and contribute to improving LLM training data quality. You will handle data labeling, RLHF ranking, prompt evaluation, and QA checks across multiple task queues.

This fully remote role emphasizes asynchronous collaboration and requires strong English comprehension, attention to rubric-driven guidelines, and the ability to maintain throughput while upholding quality standards.

Qualifications

• Mid-senior capability in remote operations, analysis, or digital production work.
• Strong English reading comprehension and clear written reasoning.
• High attention to detail; comfort working with rubrics and taxonomies.
• Ability to maintain consistent throughput while meeting quality targets.

Responsibilities

• Complete data labeling for text and image datasets across tasks.
• Perform RLHF preference ranking and pairwise comparisons.
• Evaluate prompts and grade model responses against rubrics.
• Run QA evaluation, spot checks, and consistency checks across batches.
• Apply annotation guidelines compliance and elevate edge cases.
• Support NLP tasks such as named entity recognition (NER) and taxonomy tagging.
• Contribute to content safety labeling and policy category tagging.
• Document feedback loops and update notes based on reviewer guidance.

Skills

Remote operations

Analysis

Detail oriented

Quality targets

Job description

Rex.zone seeks a remote generalist to support structured AI/ML data operations and contribute to improving LLM training data quality. You will handle data labeling, RLHF ranking, prompt evaluation, and QA checks across multiple task queues.

This fully remote role emphasizes asynchronous collaboration and requires strong English comprehension, attention to rubric-driven guidelines, and the ability to maintain throughput while upholding quality standards.