AI Conversation Context Evaluation Expert

OpenTrain AI, Inc.; United States; Hybrid; USD 55,000 - 110,000; Full time

Job summary

OpenTrain AI, Inc. is seeking contractors to evaluate conversations between people and AI models, focusing on whether enough context is gathered before recommendations are made. You will record structured findings and help keep assessments consistent.

The role involves reviewing real-world interpersonal conversations, identifying missing context, and suggesting clarifying questions. It is a three-week project with light but steady workload.

Qualifications

• 3 to 5 years in a relevant professional field such as social work, counseling, psychology, qualitative research or related areas.
• Ability to identify details that affect recommendations and note missed opportunities to clarify information.

Responsibilities

• Review conversations between people and AI models about everyday decisions.
• Identify missing context and suggest clarifying questions.
• Assess whether additional information could change AI recommendations.
• Separate decision-relevant details from descriptive information.

Skills

Interviewing

Qualitative analysis

Conversation analysis

User research

Thematic analysis

Qualitative coding

Education

Graduate degree or higher

Job description

The Work

You will evaluate conversations between people and AI models about interpersonal situations and everyday decisions. The focus is on whether the model gathered enough relevant information before making a recommendation.

You will record structured findings using evaluation guidelines and help keep assessments consistent across cases.

• Review conversations involving real-world interpersonal and everyday decisions.
• Identify missing context and suggest questions that could clarify a person's needs or circumstances.
• Assess whether additional information could change an AI recommendation.
• Separate decision-relevant details from information that is only descriptive.

What It Pays And Takes

This is a contractor project paid per task. The project lasts three weeks and requires at least 20 hours per week.

• Pay: Per task; no rate is provided in the listing.
• Schedule: 20 or more hours per week for a three-week project.
• Location: Based in the USA, Canada, or Western Europe.
• Language: Professional fluency in English.
• Experience: 3 to 5 years in a relevant professional field, including social work, counseling, therapy, psychology, qualitative research, HCI, communication, mediation, coaching, student services, or organizational consulting.
• Relevant work may include client assessment, interviewing, case review, counseling, qualitative research, conversation analysis, user research, thematic analysis, or qualitative coding.
• Qualifications may include a relevant graduate degree, professional license or credential, or advanced expertise in HCI, communication, anthropology, sociology, or qualitative research.
• You must be able to judge which details materially affect a recommendation and recognize missed opportunities to clarify incomplete information.

About AI Training Work

AI training work uses human review to help improve how artificial intelligence understands information and responds to people. In this role, your experience with assessment, interviewing, or qualitative analysis helps evaluate whether an AI system asks for and uses the context needed to give sound recommendations.