AI Conversation Quality & Context Researcher

Gramian Consulting; South Africa; On-site; ZAR 700,000 - 1,200,000; Full time

Job summary

Gramian Consultancy is seeking experienced Subject Matter Experts from psychology, social work, counseling, qualitative research, HCI, and related advisory fields to support AI conversation evaluation research.

You will review conversations between people and AI models, assess whether the AI gathered sufficient context before making recommendations, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality.

Qualifications

• 3–5 years of direct professional experience in one or more of the listed domains.
• Licensed Social Work credentials or equivalent.
• Advanced degrees in psychology, social work, or related advisory fields.
• Experience in counseling, interviewing, or client-facing professional work.
• Ability to analyze conversations systematically and identify missing context.

Responsibilities

• Review AI conversations involving interpersonal, workplace, education, family, and wellbeing topics.
• Evaluate whether the AI gathered sufficient context before providing advice.
• Identify missing information and suggest follow-up questions.
• Assess whether additional context would change recommendations.
• Document findings using structured evaluation guidelines.
• Collaborate with research teams to maintain evaluation standards.
• Complete tasks accurately within project timelines.

Skills

Analytical thinking

Interviewing

Qualitative research

Conversation analysis

Structured feedback

Education

MSW or equivalent

LCSW/LICSW/LMSW or equivalent

LMFT/LPC/LCPC or equivalent

PhD or PsyD in Clinical/Counseling/School or IO Psychology

MA/PhD in HCI, Communication, Anthropology, Sociology or Qualitative Research

Job description

Gramian Consultancy is seeking experienced Subject Matter Experts from psychology, social work, counseling, qualitative research, HCI, and related advisory fields to support AI conversation evaluation research.

You will review conversations between people and AI models, assess whether the AI gathered sufficient context before making recommendations, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality.