AI Model Evaluator & Operational QA Specialist

Crossing Hurdles; United States; Remote; CAD 60,000 - 90,000; Full time

Job summary

Crossing Hurdles is seeking a detail-oriented evaluator to audit AI models and agents across predefined scenarios in a Canadian context. This role focuses on rigorous testing, precise execution, and thorough documentation of results.

You will manage repetitive test cycles, ensure adherence to project guidelines, and contribute to building reliable evaluation datasets for AI training platforms. Strong attention to detail and consistent performance under high-volume tasks are essential.

Qualifications

• Experience with AI products, models, or AI agent evaluation and testing.
• Ability to follow complex instructions with precision and consistency.
• Exceptional attention to detail and operational thoroughness.
• Comfort working with repetitive or high-volume operational tasks.

Responsibilities

• Evaluate AI models and agents according to detailed, pre-defined scenarios and guidelines.
• Execute operational tasks to test model accuracy, reasoning, and responsiveness.
• Document findings, observations, and anomalies in structured written reports.
• Reset usage limits and repeat evaluations as required by project protocols.
• Record findings and submit evaluation datasets on AI training platforms.

Skills

AI product experience

Follow complex instructions

Attention to detail

Handling repetitive tasks

Job description

Crossing Hurdles is seeking a detail-oriented evaluator to audit AI models and agents across predefined scenarios in a Canadian context. This role focuses on rigorous testing, precise execution, and thorough documentation of results.

You will manage repetitive test cycles, ensure adherence to project guidelines, and contribute to building reliable evaluation datasets for AI training platforms. Strong attention to detail and consistent performance under high-volume tasks are essential.