Senior Data Annotation Specialist — Remote AI/ML Labeling

Rex.zone; Atlanta (GA); Remote; USD 41,328 - 68,880; Full time

Job summary

Rex.zone in Atlanta is seeking a Senior Data Annotation professional for a full-time remote role focused on improving AI/ML training pipelines through high-quality data labeling, RLHF-related evaluation, and rigorous data quality review. You will collaborate with engineering and research teams to reduce label noise, ensure consistency across modalities, and support production LLM training workflows.

With a collaborative approach, you will work with researchers and engineers to optimize labeling

Qualifications

• Mid-senior experience in data annotation with measurable quality outcomes.
• Strong understanding of training data quality, audits, and inter-annotator agreement.
• Familiarity with LLM evaluation, RLHF workflows, and prompt evaluation.
• Experience with NLP tasks such as named entity recognition and text classification.
• Strong written communication for guideline refinement and QA reporting.

Responsibilities

• Own complex data labeling tasks across modalities (text, image, mixed inputs).
• Perform RLHF ranking, preference labeling, and rubric-based evaluation.
• Execute QA evaluation audits and enforce annotation guidelines compliance.
• Adjudicate disagreements and propose guideline updates.
• Support NLP workflows including taxonomy alignment.
• Document decisions for traceability and model performance improvement.

Skills

Data annotation quality

RLHF evaluation familiarity

NLP tasks (NER, text classification)

Communication & QA reporting

Job description

Rex.zone in Atlanta is seeking a Senior Data Annotation professional for a full-time remote role focused on improving AI/ML training pipelines through high-quality data labeling, RLHF-related evaluation, and rigorous data quality review. You will collaborate with engineering and research teams to reduce label noise, ensure consistency across modalities, and support production LLM training workflows.

With a collaborative approach, you will work with researchers and engineers to optimize labeling