Remote AI Evaluation Specialist: Q&A Crafting & Testing

Crossing Hurdles; United States; Remote; CAD 28,000 - 83,000; Part time

Job summary

Crossing Hurdles is seeking an Evaluation Specialist/Recent Grad for a remote contract role. You will craft original QA pairs, perform rigorous research with source triangulation, and design multi-step questions that challenge AI models.

This position offers flexible commitment of 10-40 hours per week, compensation of $20-$60 per hour, and the chance to contribute to cutting-edge model evaluation. Strong writing, critical thinking, and self-direction are essential.

Qualifications

• Proven ability to conduct thorough, independent research and evaluate information critically.
• Exceptional attention to detail and written precision in English.
• Talent for crafting nuanced and well-structured questions.
• Strong self-direction, reliability, and accountability in remote work.
• Interest in AI and a passion for testing model capabilities.

Responsibilities

• Develop original, challenging question-and-answer pairs to test AI models.
• Conduct in-depth research with rigorous source triangulation for accuracy.
• Create multi-step questions requiring synthesis and analytical reasoning.
• Test questions against AI models, analyze outcomes, and iterate difficulty.
• Document research process with clear citations and logical explanations.
• Refine content based on reviewer input and maintain quality standards.

Skills

Research

Critical thinking

Attention to detail

Written English

Self-directed

Remote work

Job description

Crossing Hurdles is seeking an Evaluation Specialist/Recent Grad for a remote contract role. You will craft original QA pairs, perform rigorous research with source triangulation, and design multi-step questions that challenge AI models.

This position offers flexible commitment of 10-40 hours per week, compensation of $20-$60 per hour, and the chance to contribute to cutting-edge model evaluation. Strong writing, critical thinking, and self-direction are essential.