RLHF Quality Reviewer and Model Evaluator Remote Canada English C1, French C1 CAD 40 to 65 per hour

Corpshore Solutions Corporation

Canada

Híbrido

CAD 34 000 - 55 000

Tempo parcial

14 dias+

Uma candidatura completa num minuto

Currículo e carta de apresentação personalizados, prontos a enviar.

Ultrapassa os filtros ATS

Vantagens oferecidas por esta oferta de emprego

Fully remote across Canada
Flexible weekly hours (20+)
Paid guideline training per programme
Rolling contracts with steady volume
Impact on model behavior and programme
Clear route into programme leadership

Resumo da oferta

Corpshore Solutions Corporation is seeking an auditor to review RLHF annotations and adjudicate disagreements, ensuring guideline alignment and bias checks. The role involves sampling batches, reporting metrics, and providing feedback to improve future outputs.

The position is fully remote across Canada, with a minimum commitment of twenty hours per week. Prior experience in annotation or linguistic quality is valued, along with strong reasoning and reliability on deadlines.

Qualificações

  • English at C1 with rigorous written reasoning.
  • At least two years in annotation, linguistic quality, editorial work, translation review, teaching, research or a comparable judgement-based discipline.

Responsabilidades

  • Review annotator judgements, ratings and written responses for correctness.
  • Adjudicate disagreements between annotators with documented rationale for client audit.
  • Sample and audit batches, report quality metrics against thresholds.

Conhecimentos

English C1
Written reasoning
Deadline driven

Descrição da oferta de emprego

Reinforcement learning from human feedback is only as good as the judgement behind it, and at scale that judgement has to be audited. You will be the audit. You will review the work of annotators, adjudicate disagreements, and identify whether a quality failure originates in an individual, in the training, or in the guideline itself.

Fully remote across Canada, minimum twenty hours weekly.

What you will do

Review annotator preference judgements, ratings and written responses for correctness, guideline consistency and freedom from bias. Adjudicate disagreements between annotators with a documented rationale that a client can audit. Sample and audit batches, reporting quality metrics against programme thresholds. Distinguish systematic error from individual error and say which, because the remedies are entirely different. Give annotators feedback that measurably improves their next batch. Contribute to guideline refinement where the current version produces inconsistent outcomes. Support red team and safety evaluation programmes, which can involve reviewing content designed to elicit harmful model output. That work is optional, clearly flagged at assignment, and supported.

What you bring

English at C1, with rigorous written reasoning. At least two years in annotation, linguistic quality, editorial work, translation review, teaching, research or a comparable judgement-based discipline. Ability to reason carefully about ambiguous cases and to write down the reasoning. Sound judgement on safety, bias and factual accuracy. Total reliability on deadlines, since a delayed audit blocks a training run.

Nice to have

Prior RLHF or model evaluation experience. Postgraduate qualification in linguistics, philosophy, law, cognitive science or a scientific field. French at C1, which opens bilingual programmes at a premium rate. Subject depth in a specialist domain.

What we offer

Rates that rise with review tier and language scarcity. Fully remote and flexible within weekly minimums. Paid guideline training per programme. Rolling contracts with steady volume. Work that directly shapes model behaviour, and a clear route into programme leadership.

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Remote RLHF Quality Auditor & Model Evaluator
Remote RLHF Quality Auditor & Model Evaluator

Corpshore Solutions Corporation • Canadá

Híbrido
CAD 34 000 - 55 000
Fully remote across Canada
Flexible weekly hours (20+)
Paid guideline training per programme
+3
Medical and Life Sciences Annotation Specialist Remote Canada English C1, French C1 CAD 50 to 90 per hour
Medical and Life Sciences Annotation Specialist Remote Canada English C1, French C1 CAD 50 to 90 per hour

Corpshore Solutions Corporation • Canadá

Híbrido
CAD 39 000 - 58 000
Remote work across Canada
Flexible hours
Paid guideline training
STEM Data Annotation Specialist Remote Canada English C1, French C1 CAD 45 to 75 per hour
STEM Data Annotation Specialist Remote Canada English C1, French C1 CAD 45 to 75 per hour

Corpshore Solutions Corporation • Canadá

Híbrido
CAD 40 000 - 65 000
Fully remote
RLHF Evaluation Specialist for AI Training
RLHF Evaluation Specialist for AI Training

Rex.zone • Canadá

Teletrabalho
CAD 56 714 - 94 524
Senior Data Annotation & QA Lead (Remote, Full-Time)
Senior Data Annotation & QA Lead (Remote, Full-Time)

Rex.zone • Montreal

Teletrabalho
CAD 60 000 - 90 000
Legal and Regulatory Annotation Specialist, Bilingual Remote Canada English C1, French C1 CAD 55 to 95 per hour
Legal and Regulatory Annotation Specialist, Bilingual Remote Canada English C1, French C1 CAD 55 to 95 per hour

Corpshore Solutions Corporation • Canadá

Híbrido
CAD 23 000 - 36 000
Fully remote
Flexible schedule
Paid training
Remote Data Annotator
Remote Data Annotator

RemoExperts • Montreal

Teletrabalho
CAD 59 000 - 98 000
Remote Data Labeling Specialist (Canada)
Remote Data Labeling Specialist (Canada)

Rex.zone • Canadá

Teletrabalho
CAD 48 000 - 68 000
AI Trainer Jobs in Canada
AI Trainer Jobs in Canada

Rex.zone • Canadá

Teletrabalho
CAD 56 714 - 94 524
AI Evaluation Specialist (Canada)
AI Evaluation Specialist (Canada)

OpenTrain AI, Inc. • Canadá

Teletrabalho
CAD 17 000 - 24 000