Remote Senior Data Labeling Lead for AI Training Quality

Rex.zone; Detroit (MI); On-site; USD 70,000 - 110,000; Full time

Job summary

Rex.zone is seeking a Senior Data Labeling Specialist to lead high-impact data labeling and evaluation workflows for AI training data quality. This fully remote role in the United States focuses on supporting LLM training pipelines, RLHF-style tasks, and prompt evaluation while aligning with Detroit-area search intent.

The candidate will manage labeling workflows, create and iterate guidelines, conduct QA evaluations, and ensure accurate ground truth through adjudication and documentation.

Qualifications

• Mid-senior experience in data labeling/annotation operations including review and adjudication.
• Hands-on knowledge of QA methods (sampling, error taxonomies, agreement metrics).
• Ability to write clear, enforceable annotation guidelines with strong reasoning.
• Experience in NLP labeling, NER, computer vision annotation, or content safety labeling.
• Strong written communication for guideline clarity and audit trails.

Responsibilities

• Lead complex data labeling workflows for LLM evaluation, including prompt evaluation and RLHF evaluation.
• Create, maintain, and iterate annotation guidelines, taxonomies, and edge-case rules.
• Run QA evaluation loops using sampling plans, inter-annotator agreement checks, and targeted audits.
• Triage ambiguity and coordinate adjudication to maintain consistent ground truth.
• Monitor training data quality signals and provide feedback tied to model performance improvement.
• Support dataset curation, coverage analysis, and reproducible documentation for compliance.

Skills

Data labeling

Annotation operations

QA methods

Guideline writing

Communication

Job description

Rex.zone is seeking a Senior Data Labeling Specialist to lead high-impact data labeling and evaluation workflows for AI training data quality. This fully remote role in the United States focuses on supporting LLM training pipelines, RLHF-style tasks, and prompt evaluation while aligning with Detroit-area search intent.

The candidate will manage labeling workflows, create and iterate guidelines, conduct QA evaluations, and ensure accurate ground truth through adjudication and documentation.