AI Conversation Quality & Insight Analyst

Gramian Consulting; New Zealand; On-site; NZD 90,000 - 130,000; Full time

Job summary

Gramian Consultancy is seeking experienced Subject Matter Experts in psychology, social work, counseling, qualitative research, HCI, and related advisory fields to support AI conversation evaluation research.

Review conversations, assess context gathering, and provide recommendations to improve AI responses. Candidates should have 3–5+ years of relevant experience and strong analytical skills.

Qualifications

• Licenses or credentials in social work or related field are required or highly preferred.
• Experience in client assessment, interviewing, counseling, or social-work practice.
• Strong background in qualitative research, interviews, user research, or conversation analysis.
• Ability to distinguish material factors affecting recommendations from less relevant details.
• Direct advisory or client-facing experience (min. ~5 years) is desirable.

Responsibilities

• Review AI conversations involving real-world interpersonal situations and everyday decisions.
• Identify missing context and suggest follow-up questions to improve AI responses.
• Evaluate whether additional information would change AI recommendations.
• Differentiate essential information from non-essential details in conversations.
• Conduct qualitative reviews and document findings using structured guidelines.
• Assess consideration of personal, psychological, social, financial, and organizational factors.
• Identify opportunities for clarification and deeper information gathering.
• Collaborate with research teams to ensure evidence-based assessments.
• Maintain accuracy, clarity, and timely delivery of reviews.

Skills

Qualitative analysis

Conversation analysis

Research design

Critical thinking

Analytical reasoning

Structured feedback

Communication skills

Context gathering

Coaching

Mediation

Education

MSW or equivalent

Licensure: LCSW/LICSW/LMSW or equivalent

Master’s in social work or related field

PhD/PsyD in Clinical/Counseling/Social/IO Psychology

MA/PhD in HCI/Communication/Anthropology/Sociology/Qualitative Research

Job description

Gramian Consultancy is seeking experienced Subject Matter Experts in psychology, social work, counseling, qualitative research, HCI, and related advisory fields to support AI conversation evaluation research.

Review conversations, assess context gathering, and provide recommendations to improve AI responses. Candidates should have 3–5+ years of relevant experience and strong analytical skills.