AI Conversation Health Auditor

Mercor; New York (NY); Remote; USD 55,000 - 83,000; Part time

Job summary

Cincinnatus LLC places skilled behavioral health professionals with leading technology teams to evaluate AI interactions in non-crisis, everyday conversations. You will review responses across relationships, wellbeing, beliefs and personal decisions to ensure balance and safety, contributing to rubric creation and model testing.

The role emphasizes 20+ hours/week, with potential for up to 40 hours, as part of an extended workforce at a top AI lab.

Qualifications

• Degree in psychology, counseling, social work, behavioral health or related field, or equivalent experience.
• 3+ years in mental health, counseling or social services setting; licensure valued but not required.
• Ability to stay neutral and explain reasoning behind judgments.
• Familiarity with motivational interviewing and boundaries; understand cognitive distortions.
• Available to engage 20+ hours/week on weekdays; strong written skills.

Responsibilities

• Evaluate conversations between users and AI models for neutrality, appropriateness and safety.
• Identify patterns of concern and document rationale clearly.
• Develop rubrics and reference responses for balanced, supportive replies.
• Design test scenarios to probe handling of sensitive topics.
• Collaborate with researchers to keep evaluation standards consistent.

Education

Degree in psychology or related field

Clinical licensure valued (LMFT/LCSW/LPC/LMHC/PsyD/PhD)

3+ years in mental health or related setting

Ability to explain professional judgment clearly

Familiarity with counseling concepts (motivational interviewing, boundaries)

Reliable 20+ hours/week availability (weekdays)

Strong written communication skills

Job description

Cincinnatus LLC places skilled behavioral health professionals with leading technology teams to evaluate AI interactions in non-crisis, everyday conversations. You will review responses across relationships, wellbeing, beliefs and personal decisions to ensure balance and safety, contributing to rubric creation and model testing.

The role emphasizes 20+ hours/week, with potential for up to 40 hours, as part of an extended workforce at a top AI lab.