Remote AI Researcher (Contract) - Domain Expert Evaluation

Crossing Hurdles; United States; Remote; CAD 41,000 - 83,000; Part time

Job summary

Crossing Hurdles is seeking a detail-focused evaluator to design domain-specific AI assessment tasks and surface reasoning gaps. This contract role is remote, with 10-40 hours per week, ideal for researchers and experts who can verify claims against primary sources.

You will author targeted questions, assess AI outputs, and document precise rationales and feedback, using strict evaluation guidelines to ensure high-quality, replicable results across projects.

Qualifications

• Proven hands-on experience with AI model training, annotation, RLHF, or evaluation.
• Deep expertise in a core domain: history, archival or library science, law, finance, journalism, or technology/industry history.
• Advanced research skills with a track record of verifying complex claims against authoritative sources.
• Exceptional written and verbal communication skills to articulate findings and feedback.
• Demonstrated ability to precisely follow detailed instructions and evaluation frameworks.

Responsibilities

• Design nuanced, domain-specific tasks that probe AI model limits and surface reasoning flaws.
• Author questions and scenarios demanding deep subject knowledge, beyond general understanding.
• Assess AI-generated outputs, validating accuracy against authoritative sources in your area of expertise.
• Document clear rationales for model shortcomings, specifying missed reasoning, source errors, or nuances.
• Apply detailed task guidelines and evaluation rubrics consistently to each submission.

Skills

AI evaluation

Annotation

RLHF

Domain expertise

Research skills

Communication

Job description

Crossing Hurdles is seeking a detail-focused evaluator to design domain-specific AI assessment tasks and surface reasoning gaps. This contract role is remote, with 10-40 hours per week, ideal for researchers and experts who can verify claims against primary sources.

You will author targeted questions, assess AI outputs, and document precise rationales and feedback, using strict evaluation guidelines to ensure high-quality, replicable results across projects.