Remote RLHF Quality Auditor & Model Evaluator

Corpshore Solutions Corporation; Canada; Hybrid; CAD 34,000 - 55,000; Part time

Benefits offered by this job

Fully remote across Canada

Flexible weekly hours (20+)

Paid guideline training per programme

Rolling contracts with steady volume

Impact on model behavior and programme

Clear route into programme leadership

Job summary

Corpshore Solutions Corporation is seeking an auditor to review RLHF annotations and adjudicate disagreements, ensuring guideline alignment and bias checks. The role involves sampling batches, reporting metrics, and providing feedback to improve future outputs.

The position is fully remote across Canada, with a minimum commitment of twenty hours per week. Prior experience in annotation or linguistic quality is valued, along with strong reasoning and reliability on deadlines.

Qualifications

• English at C1 with rigorous written reasoning.
• At least two years in annotation, linguistic quality, editorial work, translation review, teaching, research or a comparable judgement-based discipline.

Responsibilities

• Review annotator judgements, ratings and written responses for correctness.
• Adjudicate disagreements between annotators with documented rationale for client audit.
• Sample and audit batches, report quality metrics against thresholds.

Skills

English C1

Written reasoning

Deadline driven

Job description

Corpshore Solutions Corporation is seeking an auditor to review RLHF annotations and adjudicate disagreements, ensuring guideline alignment and bias checks. The role involves sampling batches, reporting metrics, and providing feedback to improve future outputs.

The position is fully remote across Canada, with a minimum commitment of twenty hours per week. Prior experience in annotation or linguistic quality is valued, along with strong reasoning and reliability on deadlines.