Senior Data Labeling Lead for AI Training (Remote)

Rex.zone; Birmingham (AL); On-site; USD 41,328 - 68,880; Full time

Job summary

Rex.zone is hiring a Mid-Senior, full-time Senior Data Labeling professional to produce high-quality training data for AI/ML systems across NLP, computer vision, and content safety. You will label datasets used in LLM training pipelines, RLHF preference labeling, prompt evaluation, and QA evaluation to drive model performance improvements.

Join a remote US-based role to execute labeling, audits, and adjudication with emphasis on edge-case documentation and compliance with client guidelines.

Qualifications

• Production experience in senior/lead data labeling, data annotation, or evaluation workflows.
• Strong annotation guidelines compliance and consistent, auditable decision-making.
• Hands-on QA evaluation: sampling, auditing, adjudication, and root-cause analysis.
• Familiarity with RLHF, prompt evaluation, NER, and/or computer vision annotation.
• Clear written communication for edge-case documentation and stakeholder collaboration.

Responsibilities

• Execute and review complex data labeling and evaluation tasks aligned to annotation guidelines and client specifications.
• Perform RLHF-oriented preference labeling, rubric-based scoring, and prompt/response evaluation.
• Run training data quality audits, sampling checks, and adjudication to improve inter-annotator agreement.
• Annotate NLP tasks (e.g., named entity recognition) and computer vision tasks (boxes, polygons, segmentation, keypoints).
• Label content safety datasets (policy categories, severity, rationales) and document edge cases clearly.
• Partner with engineering/ops to refine taxonomies, gold sets, acceptance criteria, and error taxonomies.

Skills

Data labeling

Annotation guidelines

QA evaluation

RLHF labeling

NLP annotation

Job description

Rex.zone is hiring a Mid-Senior, full-time Senior Data Labeling professional to produce high-quality training data for AI/ML systems across NLP, computer vision, and content safety. You will label datasets used in LLM training pipelines, RLHF preference labeling, prompt evaluation, and QA evaluation to drive model performance improvements.

Join a remote US-based role to execute labeling, audits, and adjudication with emphasis on edge-case documentation and compliance with client guidelines.