Senior Data Labeling Lead AI/ML Training & Evaluation

Rex.zone; Denver (CO); On-site; USD 70,000 - 105,000; Full time

Benefits offered by this job

Remote-friendly

Job summary

Rex.zone is hiring a Senior Data Labeling Specialist to lead and execute high-quality training data workflows for AI/ML systems. You will support RLHF, LLM evaluation, QA evaluation, and annotation operations across NLP, computer vision, and content safety labeling to drive measurable model performance improvement.

Responsibilities include expert data labeling across text, image, and multimodal tasks; apply annotation guidelines; run QA workflows; and collaborate with engineering to measure

Qualifications

• Experience in data labeling/data annotation or LLM evaluation
• Strong understanding of training data quality and taxonomy design
• Experience with QA evaluation methods and calibration sessions
• Familiarity with RLHF concepts, ranking, and rubric-driven evaluation
• Ability to document decisions clearly and communicate edge cases precisely
• Comfort working remotely in a full-time schedule with measurable output targets

Responsibilities

• Perform expert data labeling across text, image, and multimodal tasks
• Execute RLHF-style preference labeling, rubric-based scoring, and LLM/prompt evaluation
• Apply annotation guidelines compliance checks and resolve edge cases
• Run QA evaluation workflows (spot checks, double-pass review, inter-annotator agreement)
• Deliver issue reports and propose guideline/rubric updates to reduce label noise
• Support content safety labeling and policy adherence evaluation
• Collaborate with engineering on sampling, task design, and quality measurement

Skills

Data labeling

RLHF evaluation

QA evaluation

Taxonomy design

Guideline compliance

Remote work

Job description

Rex.zone is hiring a Senior Data Labeling Specialist to lead and execute high-quality training data workflows for AI/ML systems. You will support RLHF, LLM evaluation, QA evaluation, and annotation operations across NLP, computer vision, and content safety labeling to drive measurable model performance improvement.

Responsibilities include expert data labeling across text, image, and multimodal tasks; apply annotation guidelines; run QA workflows; and collaborate with engineering to measure