Senior Data Annotation Lead - Remote NLP, RLHF & CV

Rex.zone; Los Angeles (CA); On-site; USD 80,000 - 120,000; Full time

Job summary

Rex.zone is hiring a Senior Data Annotation Specialist in Los Angeles for remote, full-time work. You will lead high-quality training data workflows for AI/ML systems, supporting NLP and CV annotation while collaborating with engineering to improve model outcomes.

You will create and refine annotation guidelines, run QA sampling, and execute RLHF workflows to enhance data quality and auditability across training pipelines.

Qualifications

• Mid-senior experience in data labeling, data annotation, or AI data operations.
• Expertise with annotation guidelines compliance and QA evaluation processes.
• Familiarity with RLHF, LLM evaluation, and/or prompt evaluation methodologies.
• Strong written communication for rubric design and reviewer feedback.
• Comfort working with structured schemas for NLP tasks (NER, sentiment, intent, toxicity).
• Ability to work independently in a remote full-time environment.

Responsibilities

• Lead senior annotation queues for LLM training pipelines and multimodal datasets
• Create and refine annotation guidelines, edge-case policies, and labeling taxonomies
• Run QA evaluation sampling, inter-annotator agreement checks, and error analysis
• Execute RLHF workflows (preference ranking, pairwise comparisons, rubric scoring)
• Perform prompt evaluation for helpfulness, harmlessness, and instruction following
• Deliver consistent NLP labels including NER and text classification
• Support computer vision annotation (bounding boxes, polygons, segmentation masks) as needed
• Coordinate with engineering/ops to resolve data issues, tooling gaps, and throughput bottlenecks
• Document decisions to ensure auditability for training data quality reviews

Skills

Data labeling

Annotation guidelines

QA evaluation

RLHF familiarity

Prompt evaluation

NLP labeling

CV labeling

Written communication

Independent work

Remote work

Job description

Rex.zone is hiring a Senior Data Annotation Specialist in Los Angeles for remote, full-time work. You will lead high-quality training data workflows for AI/ML systems, supporting NLP and CV annotation while collaborating with engineering to improve model outcomes.

You will create and refine annotation guidelines, run QA sampling, and execute RLHF workflows to enhance data quality and auditability across training pipelines.