Behavioral Health AI Safety & Model Evaluation (Part-Time)

Annotation Academy; United States; Remote; USD 55,000 - 83,000; Part time

Job summary

Cincinnatus LLC is seeking behavioral health experts to help evaluate how AI models respond in everyday conversations about relationships, wellbeing and beliefs. This part-time role focuses on neutrality, safety and balanced guidance, with 20–40 hours per week and a W-2 employment arrangement within the lab's extended workforce.

You will review user-model interactions, document patterns of concern, develop rubrics and reference responses, and collaborate with researchers to maintain consistent,

Qualifications

• A degree in psychology, counseling, social work, behavioral health, behavioral science, human services or a closely related field, or equivalent professional experience in the mental health field.
• 3+ years of professional experience in mental health, counseling or social services settings (therapist, counselor, clinical social worker, psychologist, etc.). Clinical licensure valued but not required.
• Demonstrated ability to remain neutral and nonjudgmental across diverse perspectives and explain professional judgments.

Responsibilities

• Evaluate conversations between users and AI models for neutrality, safety and appropriateness.
• Identify patterns of concern and document clear rationale.
• Develop rubrics, guidelines and reference responses for balanced replies.
• Design test scenarios probing sensitive but non-crisis topics.
• Collaborate with researchers to maintain evaluation standards and documentation.

Skills

Neutral judgment

Written communication

Clear written feedback

20 hours/week availability

Education

Bachelor's degree or equivalent in psychology, counseling, social work, behavioral health, behavioral science, human services

Job description

Cincinnatus LLC is seeking behavioral health experts to help evaluate how AI models respond in everyday conversations about relationships, wellbeing and beliefs. This part-time role focuses on neutrality, safety and balanced guidance, with 20–40 hours per week and a W-2 employment arrangement within the lab's extended workforce.

You will review user-model interactions, document patterns of concern, develop rubrics and reference responses, and collaborate with researchers to maintain consistent,