Remote: Entry-Level AI Data Labeler & Evaluator

Rex.zone; United States; Remote; USD 41,328 - 68,880; Full time

Job summary

A leading AI training company is seeking a Remote Annotator to support human-in-the-loop AI training workflows for large language models. This role involves reviewing labeled datasets, performing evaluations for helpfulness and safety, and ensuring quality in training data. Strong written English and attention to detail are essential. Competitive hourly compensation of $30–$50 is offered, making it suitable for detail-oriented individuals interested in AI training.

Qualifications

• Strong written English and structured reasoning.
• Ability to follow detailed annotation guidelines.
• Comfort using spreadsheets and web tools.
• Familiarity with model evaluation and prompt patterns.
• Attention to detail with QA evaluation.
• Comfort handling sensitive content for safety labeling.

Responsibilities

• Create and review labeled datasets for various use cases.
• Perform RLHF preference ranking and scoring.
• Run evaluations for helpfulness and safety.
• Execute content safety labeling for various policies.
• Conduct QA evaluation via audits and tracking.
• Track metrics impacting LLM training.

Skills

Strong written English

Detailed annotation guidelines

Spreadsheets and web tools

Model evaluation familiarity

Attention to detail

Content safety handling

Job description

A leading AI training company is seeking a Remote Annotator to support human-in-the-loop AI training workflows for large language models. This role involves reviewing labeled datasets, performing evaluations for helpfulness and safety, and ensuring quality in training data. Strong written English and attention to detail are essential. Competitive hourly compensation of $30–$50 is offered, making it suitable for detail-oriented individuals interested in AI training.