Remote RLHF Quality Auditor & Model Evaluator

Corpshore Solutions Corporation

Canada

Hybrid

CAD 34,000 - 55,000

Part time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Fully remote across Canada
Flexible weekly hours (20+)
Paid guideline training per programme
Rolling contracts with steady volume
Impact on model behavior and programme
Clear route into programme leadership

Job summary

Corpshore Solutions Corporation is seeking an auditor to review RLHF annotations and adjudicate disagreements, ensuring guideline alignment and bias checks. The role involves sampling batches, reporting metrics, and providing feedback to improve future outputs.

The position is fully remote across Canada, with a minimum commitment of twenty hours per week. Prior experience in annotation or linguistic quality is valued, along with strong reasoning and reliability on deadlines.

Qualifications

  • English at C1 with rigorous written reasoning.
  • At least two years in annotation, linguistic quality, editorial work, translation review, teaching, research or a comparable judgement-based discipline.

Responsibilities

  • Review annotator judgements, ratings and written responses for correctness.
  • Adjudicate disagreements between annotators with documented rationale for client audit.
  • Sample and audit batches, report quality metrics against thresholds.

Skills

English C1
Written reasoning
Deadline driven

Job description

Corpshore Solutions Corporation is seeking an auditor to review RLHF annotations and adjudicate disagreements, ensuring guideline alignment and bias checks. The role involves sampling batches, reporting metrics, and providing feedback to improve future outputs.

The position is fully remote across Canada, with a minimum commitment of twenty hours per week. Prior experience in annotation or linguistic quality is valued, along with strong reasoning and reliability on deadlines.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

RLHF Quality Reviewer and Model Evaluator Remote Canada English C1, French C1 CAD 40 to 65 per hour
RLHF Quality Reviewer and Model Evaluator Remote Canada English C1, French C1 CAD 40 to 65 per hour

Corpshore Solutions Corporation • Canada

Hybrid
CAD 34,000 - 55,000
Fully remote across Canada
Flexible weekly hours (20+)
Paid guideline training per programme
+3
Remote Data Labeling Specialist (Canada)
Remote Data Labeling Specialist (Canada)

Rex.zone • Canada

Remote
CAD 48,000 - 68,000
Remote Data Annotator Jobs Toronto
Remote Data Annotator Jobs Toronto

Rex.zone • Toronto

Remote
CAD 42,000 - 62,000
Remote Clinical AI Safety & Annotation Specialist
Remote Clinical AI Safety & Annotation Specialist

Corpshore Solutions Corporation • Canada

Hybrid
CAD 39,000 - 58,000
Remote work across Canada
Flexible hours
Paid guideline training
RLHF Evaluation Specialist for AI Training
RLHF Evaluation Specialist for AI Training

Rex.zone • Canada

Remote
CAD 56,714 - 94,524
Remote Data Annotator
Remote Data Annotator

RemoExperts • Montreal

Remote
CAD 59,000 - 98,000
Senior Data Annotation & QA Lead (Remote, Full-Time)
Senior Data Annotation & QA Lead (Remote, Full-Time)

Rex.zone • Montreal

Remote
CAD 60,000 - 90,000
Remote Data Annotation Specialist: Generalist (Part-Time)
Remote Data Annotation Specialist: Generalist (Part-Time)

Avenue.io Inc. • Toronto, Montreal

Remote
CAD 19,000 - 24,000
Annotation Quality Editor (Remote)
Annotation Quality Editor (Remote)

Work Vista • Canada

Remote
CAD 28,474 - 34,169
Flexible hours
100% remote work
Opportunity to work up to 40 hours weekly
Remote Data Labeling Jobs in Ottawa
Remote Data Labeling Jobs in Ottawa

Rex.zone • Ottawa

Remote
CAD 59,000 - 98,000