AI Behavioral Health Evaluator (Part-Time, 20–40 hrs)

Mercor; New York (NY); On-site; USD 83,000 - 124,000; Part time

Job summary

Cincinnatus LLC is placing behavioral health experts to help a leading AI lab evaluate how its models respond in everyday conversations about relationships, wellbeing and beliefs. This part-time role requires 20 hours per week (up to 40) and involves reviewing model outputs, identifying improvements and contributing to balanced, nonjudgmental guidance.

You'll work within a W-2 employment framework, with the opportunity to be placed at a top AI lab as part of the extended workforce, collaborating

Qualifications

• A degree in psychology, counseling, social work, behavioral health, or related field, or equivalent professional experience in mental health.
• 3+ years of professional experience in mental health, counseling or social services; licensure valued but not required.
• Demonstrated ability to remain neutral across diverse perspectives and explain professional judgments.
• Familiarity with sycophancy, cognitive distortions, healthy boundaries and client-centered approaches such as motivational interviewing.
• Available for at least 20 hours/week during weekdays.
• Strong written communication and ability to deliver precise, well-structured feedback.

Responsibilities

• Evaluate conversations between users and AI models for neutrality, appropriateness and safety.
• Identify patterns of concern and document rationale.
• Develop rubrics, guidelines and reference responses for balanced replies.
• Design test scenarios that probe handling of sensitive topics.
• Collaborate with researchers to maintain consistent evaluation standards.

Skills

Neutrality

Written communication

Clear feedback

Time commitment

Education

Degree in psychology or related field

3+ years in mental health or related field

Licensure valued but not required

Job description

Cincinnatus LLC is placing behavioral health experts to help a leading AI lab evaluate how its models respond in everyday conversations about relationships, wellbeing and beliefs. This part-time role requires 20 hours per week (up to 40) and involves reviewing model outputs, identifying improvements and contributing to balanced, nonjudgmental guidance.

You'll work within a W-2 employment framework, with the opportunity to be placed at a top AI lab as part of the extended workforce, collaborating