Remote Data Annotator Jobs in New York

Rex.zone; New York (NY); Remote; USD 42,000 - 62,000; Full time

Job summary

Rex.zone is seeking a Remote Data Annotator to label and evaluate datasets used to train ML models. You will contribute to RLHF feedback loops and ensure data quality across NLP, CV, and LLM workflows.

The role emphasizes high-precision labeling, adherence to guidelines, and collaboration with QA, reviewers, and data operations teams from a remote setup in New York.

Qualifications

• Professional experience in data annotation or data labeling.
• Strong written English comprehension and attention to detail.
• Ability to follow detailed guidelines with high precision and consistency.

Responsibilities

• Produce high-accuracy labels for text, images, and multimodal inputs.
• Perform RLHF-style preference ranking and prompt evaluation for LLMs.
• Complete QA evaluation checks, audits, and consistency reviews.
• Apply named entity recognition and taxonomy tagging.
• Conduct content safety labeling for policy and harm categories.
• Document edge cases and guideline gaps to improve labeling instructions.
• Collaborate asynchronously with reviewers, QA, and data operations teams.

Skills

Data labeling experience

Strong written English

Attention to detail

Tools

Annotation tools

CV annotation tools

Job description

About The Role

As a Remote Data Annotator supporting New York-based programs, you will label and evaluate datasets used to train and validate machine learning models. Your work improves training data quality, supports RLHF feedback loops, and drives model performance improvement across NLP, computer vision, and LLM workflows.

What You Will Do

• Produce high-accuracy labels for text, images, and multimodal inputs
• Perform RLHF-style preference ranking and prompt evaluation for LLMs
• Complete QA evaluation checks, audits, and consistency reviews
• Apply named entity recognition and taxonomy tagging
• Conduct content safety labeling for policy and harm categories
• Document edge cases and guideline gaps to improve labeling instructions
• Collaborate asynchronously with reviewers, QA, and data operations teams

Core Workflows You May Support

• Large language model evaluation and prompt-response grading
• Classification and ranking tasks; training data curation
• Computer vision annotation (bounding boxes, polygons, keypoints)
• Conversation quality scoring and safety policy enforcement

Required Qualifications

• Professional experience in data annotation or data labeling
• Strong written English comprehension and attention to detail
• Ability to follow detailed guidelines with high precision and consistency

Preferred Qualifications

• Experience with RLHF, LLM training pipelines, prompt evaluation, or QA evaluation
• Familiarity with NER, content safety labeling, and CV annotation tools

Remote Work Notes (New York)

These roles are Remote and can be performed from New York. Some projects may require availability aligned with US business hours or structured QA review cycles.