Remote AI Data Annotation & Evaluation Specialist

Rex.zone; Detroit (MI); On-site; USD 41,328 - 68,880; Full time

Job summary

Rex.zone is seeking a Remote AI Data Annotation Specialist to support end-to-end LLM training pipelines and model evaluation datasets. You will apply detailed rubrics to improve data quality and model reliability, working asynchronously from home in the US.

You will label text, image, and multimodal data, perform RLHF-style ranking, prompt evaluation, and NLP/CV annotation tasks, with a focus on accuracy and thorough documentation. This role is fully remote.

Qualifications

• Mid-Senior experience in AI data annotation, data labeling operations, or model evaluation.
• Ability to follow complex rubrics and provide consistent judgments.
• Familiarity with NLP concepts (e.g., NER, classification) and comfort with CV annotation concepts.
• Strong written communication, attention to detail, and reliable remote work habits.

Responsibilities

• Label and review text, image, and multimodal datasets to meet training data quality thresholds.
• Execute RLHF-style preference ranking and rubric-based evaluations of model outputs.
• Perform prompt evaluation and QA evaluation (calibration, audit sampling, consistency checks).
• Annotate NLP tasks such as named entity recognition, taxonomy mapping, and document classification.
• Support computer vision annotation (bounding boxes, polygons/segmentation, keypoints, attributes).
• Complete content safety labeling using policy-based decision rules.
• Document edge cases, resolve disagreements, and improve annotation guidelines compliance.

Skills

AI data annotation

Data labeling

Model evaluation

NLP concepts (NER, classification)

CV annotation concepts

Written communication

Attention to detail

Remote work

Job description

Rex.zone is seeking a Remote AI Data Annotation Specialist to support end-to-end LLM training pipelines and model evaluation datasets. You will apply detailed rubrics to improve data quality and model reliability, working asynchronously from home in the US.

You will label text, image, and multimodal data, perform RLHF-style ranking, prompt evaluation, and NLP/CV annotation tasks, with a focus on accuracy and thorough documentation. This role is fully remote.