Remote AI Domain Evaluation Specialist (Contractor)

Remotebridge; United States; Remote; USD 83,000 - 165,000; Part time

Job summary

Remotebridge seeks a Domain Contributor - Evaluation Specialist for a remote contractor role (~15 hrs/week). You will design nuanced, domain-specific tasks and assess AI outputs against primary sources, documenting precise rationales for shortcomings.

Preferred candidates have graduate-level research experience and strong written/verbal communication, with a track record in adversarial task design or insider-domain expertise.

Qualifications

• Hands-on experience with AI model evaluation and annotation.
• Deep domain expertise in one of the listed fields.
• Proven ability to verify complex claims against sources.

Responsibilities

• Design domain-specific tasks to probe AI limits and surface reasoning flaws.
• Author questions and scenarios requiring deep domain knowledge.
• Assess AI outputs for accuracy against primary sources.
• Document clear rationales for model shortcomings with specifics.
• Apply detailed guidelines and evaluation rubrics consistently.
• Submit deliverables independently on a rolling basis.

Skills

AI model evaluation

RLHF experience

Advanced research

Precise instruction following

Strong communication

Education

Graduate-level research

Job description

Remotebridge seeks a Domain Contributor - Evaluation Specialist for a remote contractor role (~15 hrs/week). You will design nuanced, domain-specific tasks and assess AI outputs against primary sources, documenting precise rationales for shortcomings.

Preferred candidates have graduate-level research experience and strong written/verbal communication, with a track record in adversarial task design or insider-domain expertise.