Remote Senior AI Data Annotation Specialist — RLHF & CV QA

Rex.zone

Atlanta (GA)

Remote

USD 41,328 - 68,880

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Rex.zone is hiring a Senior AI Data Annotation Specialist to create, review, and audit high-quality training data for modern AI systems, including LLMs and multimodal models. You will execute annotation tasks, run QA workflows, and support RLHF through ratings and critiques that improve model performance.

The role is remote, full-time, with collaboration across distributed teams in the US. You will work on NLP and computer vision annotation, policy-based evaluation, and data quality audits,

Qualifications

  • Experience delivering high-volume, high-precision AI data annotation.
  • Strong English comprehension and writing for prompt evaluation and critique.
  • Familiarity with RLHF concepts (pairwise ranking, preference data).
  • Experience with QA evaluation processes and reviewer calibration.

Responsibilities

  • Produce accurate labels while maintaining annotation guidelines compliance.
  • Perform QA evaluation, second-pass reviews, and sampling-based audits.
  • Calibrate with reviewers to maintain consistency and inter-annotator agreement.
  • Document edge cases, resolve disagreements, and propose guideline updates.
  • Partner with data operations and engineering to improve tools and workflows and report systematic failure modes observed during evaluations.

Skills

High-volume annotation
RLHF concepts
English writing
QA evaluation
Reviewer calibration
NLP annotation
Computer vision annotation

Job description

Rex.zone is hiring a Senior AI Data Annotation Specialist to create, review, and audit high-quality training data for modern AI systems, including LLMs and multimodal models. You will execute annotation tasks, run QA workflows, and support RLHF through ratings and critiques that improve model performance.

The role is remote, full-time, with collaboration across distributed teams in the US. You will work on NLP and computer vision annotation, policy-based evaluation, and data quality audits,

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Data Annotation & Evaluation Specialist (Remote)
Senior AI Data Annotation & Evaluation Specialist (Remote)

Rex.zone • Seattle (WA)

Remote
USD 41,328 - 68,880
Remote Senior AI Data Labeler - RLHF, NER & CV Annotation
Remote Senior AI Data Labeler - RLHF, NER & CV Annotation

Rex.zone • Austin (TX)

Remote
USD 65,000 - 95,000
Senior Data Annotation & RLHF QA Specialist
Senior Data Annotation & RLHF QA Specialist

Rex.zone • Boston (MA)

Remote
USD 70,000 - 110,000
Senior AI Data Annotation & RLHF Evaluator (Remote)
Senior AI Data Annotation & RLHF Evaluator (Remote)

Rex.zone • Miami (FL)

Remote
USD 55,000 - 85,000
Remote AI Data Annotation & Evaluation Specialist
Remote AI Data Annotation & Evaluation Specialist

Rex.zone • Detroit (MI)

Remote
USD 41,328 - 68,880
Remote AI Data Annotator for NLP, Vision & RLHF
Remote AI Data Annotator for NLP, Vision & RLHF

Rex.zone • Miami (FL)

Remote
USD 41,328 - 68,880
Senior Data Annotator: AI Evaluation & QA Lead (Remote)
Senior Data Annotator: AI Evaluation & QA Lead (Remote)

Rex.zone • Phoenix (AZ)

Remote
USD 41,328 - 68,880
Remote Senior Data Annotation Lead for AI Training & QA
Remote Senior Data Annotation Lead for AI Training & QA

Rex.zone • Denver (CO)

Remote
USD 41,328 - 68,880
Senior Data Annotator - Remote AI Training & Evaluation
Senior Data Annotator - Remote AI Training & Evaluation

Rex.zone • Seattle (WA)

Remote
USD 41,328 - 68,880
Remote Data Annotation Specialist for AI Training & QA
Remote Data Annotation Specialist for AI Training & QA

Rex.zone • Denver (CO)

Remote
USD 42,000 - 54,000