Domain Contributor - Evaluation Specialist

Remotebridge; United States; A distancia; USD 83.000 - 165.000; A tiempo parcial

Descripción de la vacante

Remotebridge seeks a Domain Contributor - Evaluation Specialist for a remote contractor role (~15 hrs/week). You will design nuanced, domain-specific tasks and assess AI outputs against primary sources, documenting precise rationales for shortcomings.

Preferred candidates have graduate-level research experience and strong written/verbal communication, with a track record in adversarial task design or insider-domain expertise.

Formación

• Hands-on experience with AI model evaluation and annotation.
• Deep domain expertise in one of the listed fields.
• Proven ability to verify complex claims against sources.

Responsabilidades

• Design domain-specific tasks to probe AI limits and surface reasoning flaws.
• Author questions and scenarios requiring deep domain knowledge.
• Assess AI outputs for accuracy against primary sources.
• Document clear rationales for model shortcomings with specifics.
• Apply detailed guidelines and evaluation rubrics consistently.
• Submit deliverables independently on a rolling basis.

Conocimientos

AI model evaluation

RLHF experience

Advanced research

Precise instruction following

Strong communication

Educación

Graduate-level research

Descripción del empleo

Role Title: Domain Contributor - Evaluation Specialist

Role Type: Contractor (~15 hrs/week)

Location: Remote

Scope of Work

• Design nuanced, domain-specific tasks that probe AI model limits and surface reasoning flaws.
• Author questions and scenarios demanding deep subject knowledge, beyond general understanding.
• Assess AI-generated outputs, validating accuracy against authoritative, primary sources in your area of expertise.
• Document clear rationales for model shortcomings, specifying missed reasoning, source errors, or nuances.
• Apply detailed task guidelines and evaluation rubrics consistently to each submission.
• Submit completed deliverables independently on a rolling basis, meeting project requirements for written and verbal clarity.

Preferred Qualifications

• Proven hands-on experience with AI model training, annotation, RLHF, or evaluation.
• Deep expertise in a core domain: history, archival or library science, law (paralegal, legal academic, compliance), finance (equity research, PE/VC, forensic accounting), journalism, or technology/industry history.
• Advanced research skills with a track record of verifying complex claims against authoritative sources.
• Exceptional written and verbal communication skills to articulate findings and feedback.
• Demonstrated ability to precisely follow detailed instructions and evaluation frameworks.
• Background in adversarial or “stomp the model” task design is advantageous.
• Graduate-level research, experience at a Fortune 100 company, or professional fluency in a non-English language a plus.

Compensation Structure

Compensation is output-based; experts are paid per task that meets the project specifications. The time required to complete work may vary depending on the expert’s experience and workflow. Minimum submission requirements apply. Experts must submit a minimum of tasks per week.

Start Timeline & Availability

We typically fill roles within 48 hours and are looking for experts ready to jump in right away. If selected, we expect you to start your first tasks within 24–48 hours of completing onboarding.