Remote AI Conversation Evaluation SME

Dover; United States; Teletrabalho; USD 70 000 - 100 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Fully remote

Pay-per-task (PPT)

USD payments

Resumo da oferta

Careerflow Human Data Labs is seeking Subject Matter Experts with backgrounds in psychology, social work, counseling, qualitative research, or related advisory fields to support AI conversation evaluation research.

You will review AI conversations between participants and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality.

Qualificações

• 3–5 years of direct practice relevant qualifications and professional experience in at least one of the following areas: social work, counseling, psychology, qualitative research, or advisory fields.
• Experience with client assessment, case review, interviewing, counseling, or related client-facing work.

Responsabilidades

• Review AI conversations involving real-world interpersonal and everyday decision-making scenarios.
• Identify missing context and recommend relevant follow-up questions.
• Evaluate whether additional information would have changed the AI's recommendations.
• Distinguish between critical (load-bearing) and non-essential information.
• Conduct qualitative reviews and document findings using structured evaluation guidelines.
• Collaborate with research teams to ensure consistent, high-quality assessments.

Descrição da oferta de emprego

Careerflow Human Data Labs is seeking Subject Matter Experts with backgrounds in psychology, social work, counseling, qualitative research, or related advisory fields to support AI conversation evaluation research.

You will review AI conversations between participants and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality.