Behavioral Health Expert – AI Model Evaluation

Mercor; Atlanta (GA); On-site; USD 55,000 - 96,000; Part time

Job summary

Cincinnatus LLC is seeking behavioral health experts to review AI-user conversations on relationships, emotional wellbeing, and beliefs, ensuring responses remain neutral, safe, and appropriately bounded. The role requires a psychology background and 3+ years in mental health settings, with strong written feedback skills.

The position is part-time (20–40 hours/week) with potential to scale up, and operates as a W-2 employment through Cincinnatus LLC, with placement at a leading AI lab as part of

Qualifications

• Degree in psychology, counseling, social work, behavioral health, or closely related field or equivalent professional experience.
• 3+ years in mental health, counseling or social services setting; licensure valued but not required.
• Demonstrated neutral, nonjudgmental stance across diverse perspectives and clear reasoning behind judgments.
• Familiar with concepts like cognitive distortions, healthy boundaries, client-centered approaches like motivational interviewing.
• Ability to engage for at least 20 hours/week during weekdays.
• Strong written communication for precise feedback.

Responsibilities

• Evaluate conversations between users and AI models for neutrality, safety and appropriateness.
• Identify patterns of concern and document them with rationale.
• Develop rubrics and reference responses grounded in counseling practices.
• Design test scenarios to probe handling of sensitive topics.
• Collaborate with researchers to maintain evaluation standards.

Skills

Psychology

Counseling experience

Written communication

Nonjudgmental stance

Education

Psychology degree

Counseling credential

Job description

Cincinnatus LLC is seeking behavioral health experts to review AI-user conversations on relationships, emotional wellbeing, and beliefs, ensuring responses remain neutral, safe, and appropriately bounded. The role requires a psychology background and 3+ years in mental health settings, with strong written feedback skills.

The position is part-time (20–40 hours/week) with potential to scale up, and operates as a W-2 employment through Cincinnatus LLC, with placement at a leading AI lab as part of