Remote AI Data Labeling & Evaluation Specialist

Rex.zone; Los Angeles (CA); On-site; USD 42,000 - 64,000; Full time

Benefits offered by this job

Remote-first

Flexible schedule

Distributed teams

Job summary

Rex.zone is seeking a Remote Data Annotation Specialist to support training data creation and evaluation for modern AI systems. You will label text, images, audio, and conversations used in LLM training pipelines, RLHF workflows, and model evaluation.

This full-time, remote-first role offers flexible scheduling and collaboration with distributed teams across time zones, emphasizing accuracy, consistency, and clear edge-case documentation.

Qualifications

• Mid-Senior experience in data annotation, data labeling, QA evaluation, or model evaluation.
• Strong attention to detail and consistent guideline-based decision-making.
• Comfort working with ambiguity; strong written communication for documenting edge cases.
• Reliable remote work habits and time management.

Responsibilities

• Perform data labeling for NLP, computer vision annotation, and multimodal (text-image) tasks.
• Execute RLHF-style ranking and preference judgments to improve model alignment.
• Run LLM evaluation tasks (rubric-based scoring, prompt-response grading, error categorization).
• Apply content safety labeling policies (toxicity, self-harm, harassment, sensitive attributes).
• Conduct QA evaluation and audits; track issue/disagreement rates and improve consistency.
• Follow detailed annotation guidelines, document edge cases, and elevate ambiguous cases.

Skills

Data annotation

QA evaluation

Model evaluation

Attention to detail

Guideline-based decisions

Remote work

Time management

Written communication

Job description

Rex.zone is seeking a Remote Data Annotation Specialist to support training data creation and evaluation for modern AI systems. You will label text, images, audio, and conversations used in LLM training pipelines, RLHF workflows, and model evaluation.

This full-time, remote-first role offers flexible scheduling and collaboration with distributed teams across time zones, emphasizing accuracy, consistency, and clear edge-case documentation.