Remote AI Data Annotation & Model Evaluation Specialist

Bilinguallink; Victoriaville; Remote; CAD 39,000 - 44,000; Part time

Job summary

Cohere is seeking a part-time Data Annotation Specialist to review and improve AI model outputs across text, image, and structured data. You will rank responses, create high-quality prompts and exemplars, and help build evaluation datasets with clear justification of judgments.

This independent contractor role is remote within Canada, requires 16 hours per week, and offers a CAD $30/hour contract rate. You will work with diverse data types, including JSON and Markdown, to support model safety

Qualifications

• 1+ years in AI data annotation, LLM evaluation, content moderation, or related analytical role.
• Ability to apply guidelines to complex content with nuanced judgment.
• Comfort with ambiguity and edge cases, avoiding guessing.
• Attention to detail to maintain consistency in high-volume tasks.
• Experience with JSON, CSV/TSV, Markdown, XML, and YAML is a plus.

Responsibilities

• Evaluate and rank model outputs with clear justifications.
• Stress-test and break models to surface failure modes.
• Create datasets with prompts, responses, and exemplars.
• Develop rubrics and taxonomies for grading criteria.
• Annotate and correct multimodal data (text, image, structured).
• Calibrate annotators and ensure inter-annotator agreement.
• Adapt to evolving tasks and report quality trends.

Skills

Data annotation

LLM evaluation

Quality assurance

Tools

JSON

CSV/TSV

Markdown

XML

YAML

Job description

Cohere is seeking a part-time Data Annotation Specialist to review and improve AI model outputs across text, image, and structured data. You will rank responses, create high-quality prompts and exemplars, and help build evaluation datasets with clear justification of judgments.

This independent contractor role is remote within Canada, requires 16 hours per week, and offers a CAD $30/hour contract rate. You will work with diverse data types, including JSON and Markdown, to support model safety