Senior AI Data Annotation Lead (Remote | Full-Time)

Rex.zone; United States; On-site; USD 60,000 - 90,000; Full time

Job summary

Rex.zone is hiring a Senior AI Data Annotation specialist to own labeling workflows that improve data quality for LLM training and model performance. You will lead end-to-end labeling programs, write guidelines, run calibration sessions, and deliver high-signal datasets for supervised fine-tuning.

The role is remote (US), full-time, collaborating with cross-functional teams. You will perform QA, adjudicate edge cases, and drive rubric-based evaluation and auditability across NLP, content safety,

Qualifications

• 3+ years in AI data annotation, labeling, or QA evaluation for ML datasets.
• Experience with LLM evaluation, RLHF concepts, or prompting and rubric design.
• Strong technical writing for guidelines and decision logs.

Responsibilities

• Lead end-to-end labeling programs with measurable acceptance criteria.
• Design and maintain annotation guidelines and compliance checks.
• Run rubric-based LLM and prompt evaluation (helpfulness, factuality, instruction following).
• Support RLHF preference labeling calibration and reviewer training.
• Execute QA sampling plans, error taxonomies, and root-cause analysis.
• Coordinate with engineering and research on labeling schemas, dataset specs, and versioning.
• Optimize throughput while preserving precision/recall targets.
• Document decisions for auditability and reproducibility.

Skills

AI data labeling

QA evaluation for ML datasets

Rubric design & guidelines writing

Remote collaboration

Job description

Rex.zone is hiring a Senior AI Data Annotation specialist to own labeling workflows that improve data quality for LLM training and model performance. You will lead end-to-end labeling programs, write guidelines, run calibration sessions, and deliver high-signal datasets for supervised fine-tuning.

The role is remote (US), full-time, collaborating with cross-functional teams. You will perform QA, adjudicate edge cases, and drive rubric-based evaluation and auditability across NLP, content safety,