Remote Data Annotation Specialist for AI/ML Training

Rex.zone

California

Remote

USD 42,000 - 66,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Rex.zone offers remote, full-time data annotation work supporting AI/ML and LLM training pipelines. You will create high-quality labeled datasets and human feedback signals, including RLHF ranking, prompt evaluation, and QA evaluation, while maintaining strict annotation guidelines compliance and training data quality.

You will adapt to asynchronous review cycles, document decisions thoroughly, and contribute to ongoing improvements in annotation quality and model performance.

Qualifications

  • Experience with structured labeling workflows and web-based annotation tools.
  • Strong written reasoning and ability to follow detailed guidelines with high precision.
  • Comfort working asynchronously with reviewer feedback loops and QA processes.

Responsibilities

  • Label and categorize text, image, and multimodal data using defined taxonomies and policies.
  • Perform RLHF pairwise preference ranking and write clear rationales/justifications.
  • Execute prompt evaluation and response quality scoring for LLM evaluation.
  • Run QA evaluation audits, calibration tasks, and gold-set checks to improve consistency.
  • Complete named entity recognition (NER) and span labeling for NLP datasets.
  • Support content safety labeling and sensitive content handling per policy.
  • Maintain throughput and accuracy targets with detailed decision logs for reviewer traceability.

Skills

Structured labeling workflows
Web-based annotation tools
Attention to detail
Strong written reasoning

Tools

Annotation tools

Job description

About The Role

Rex.zone connects annotators with remote, full-time data annotation work supporting AI/ML and LLM training pipelines. You will create high-quality labeled datasets and human feedback signals, including RLHF preference ranking, prompt evaluation, and QA evaluation, while maintaining strict annotation guidelines compliance and training data quality.

About The Role

Rex.zone connects annotators with remote, full-time data annotation work supporting AI/ML and LLM training pipelines. You will create high-quality labeled datasets and human feedback signals, including RLHF preference ranking, prompt evaluation, and QA evaluation, while maintaining strict annotation guidelines compliance and training data quality.

What You Will Do

  • Label and categorize text, image, and multimodal data using defined taxonomies and policies
  • Perform RLHF pairwise preference ranking and write clear rationales/justifications
  • Execute prompt evaluation and response quality scoring for LLM evaluation
  • Run QA evaluation audits, calibration tasks, and gold-set checks to improve consistency
  • Complete named entity recognition (NER) and span labeling for NLP datasets
  • Support content safety labeling and sensitive content handling per policy
  • Maintain throughput and accuracy targets with detailed decision logs for reviewer traceability

Requirements

  • Experience with structured labeling workflows and web-based annotation tools
  • Strong written reasoning and ability to follow detailed guidelines with high precision
  • Comfort working asynchronously with reviewer feedback loops and QA processes
  • Preferred: RLHF, prompt evaluation, LLM evaluation, NER, or computer vision annotation exposure

Quality & Performance

You will be evaluated on guideline adherence, inter-annotator agreement, edge-case handling, and overall training data quality. Expect structured QA cycles, periodic calibration, and reviewer feedback to support measurable model performance improvement.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Data Annotator Jobs Manchester
Remote Data Annotator Jobs Manchester

RemoExperts • California

Remote
USD 42,000 - 66,000
Entry Level Data Labeling Specialist (United States, Remote)
Entry Level Data Labeling Specialist (United States, Remote)

Rex.zone • United States

On-site
USD 32,000 - 52,000
Remote AI/ML Data Annotation Engineer
Remote AI/ML Data Annotation Engineer

Rex.zone • United States

Remote
USD 41,328 - 68,880
Remote Data Annotation Jobs in Denver
Remote Data Annotation Jobs in Denver

Rex.zone • Denver (CO)

Remote
USD 42,000 - 54,000
Remote Data Annotation Jobs Dublin
Remote Data Annotation Jobs Dublin

Rex.zone • United States

Remote
USD 41,000 - 69,000
Remote Data Annotation Specialist for AI Training & QA
Remote Data Annotation Specialist for AI Training & QA

Rex.zone • Denver (CO)

Remote
USD 42,000 - 54,000
Remote Data Annotation Jobs Amsterdam
Remote Data Annotation Jobs Amsterdam

Rex.zone • United States

Remote
USD 41,000 - 69,000
Remote Data Labeling Jobs Toronto
Remote Data Labeling Jobs Toronto

Rex.zone • United States

Remote
USD 41,000 - 69,000
Remote AI Data Labeling & QA Specialist
Remote AI Data Labeling & QA Specialist

Rex.zone • United States

On-site
USD 41,000 - 69,000
Remote Data Labeling Specialist for AI & LLM QA
Remote Data Labeling Specialist for AI & LLM QA

Rex.zone • United States

Remote
USD 41,000 - 69,000