Senior Data Annotator - Remote AI QA & LLM Evaluation

Rex.zone; Miami (FL); On-site; USD 41,328 - 68,880; Full time

Job summary

Rex.zone is hiring mid-senior data annotators to support US-based AI/ML projects across NLP, LLM evaluation, RLHF, content safety, and computer vision. You will produce and review high-quality labeled data, ensuring strict annotation guidelines compliance and measurable model performance improvement.

Fully remote role aligned with Miami job-search intent, supporting US-based projects and competitive hourly compensation of $30-$50.

Qualifications

• Mid-senior level data annotation experience.
• Ability to follow guidelines under ambiguity.
• Experience with QA methods and quality metrics.
• Familiarity with RLHF-style judgments and LLM evaluation concepts.

Responsibilities

• Create and review training data for supervised learning and RLHF pipelines (including preference ranking).
• Perform LLM evaluation and prompt evaluation using clear rubrics.
• Execute QA evaluation: spot checks, blind review, adjudication, and calibration sessions.
• Track inter-annotator agreement and document edge cases and decision rationales.
• Label tasks such as named entity recognition (NER), text classification, and content safety labeling.
• Support computer vision annotation (bounding boxes, polygons, keypoints) when needed.

Skills

Data annotation experience

Quality ownership

Reading comprehension

RLHF/LLM evaluation knowledge

Job description

Rex.zone is hiring mid-senior data annotators to support US-based AI/ML projects across NLP, LLM evaluation, RLHF, content safety, and computer vision. You will produce and review high-quality labeled data, ensuring strict annotation guidelines compliance and measurable model performance improvement.

Fully remote role aligned with Miami job-search intent, supporting US-based projects and competitive hourly compensation of $30-$50.