Remote AI Conversation Quality Analyst & Context Specialist

Verita AI; United States; Remote; USD 32,000 - 36,000; Full time

Benefits offered by this job

Fully remote

Paid weekly

4-week contractor engagement

Job summary

Verita AI is seeking subject matter experts to review AI conversations and assess whether sufficient context was gathered before recommendations were made. You will identify missed opportunities for clarification and provide structured feedback to improve AI systems.

The role focuses on rigorous qualitative evaluation and documentation. Successful candidates will have licensure or advanced degrees in psychology, social work, or related fields, plus 5+ years of advisory or consulting experience.

Qualifications

• Must have one of the listed advanced licensures or degrees in psychology, social work, or related fields.
• Candidates should have 5+ years of advisory, coaching, or client-facing work if using related credentials.
• Strong analytical thinking and qualitative evaluation ability required.
• Excellent attention to detail and ability to apply evaluation guidelines.
• Comfort with spreadsheets, annotation platforms, or labeling tools.
• Strong written communication and documentation skills.
• Ability to evaluate nuanced conversations objectively across contexts.

Responsibilities

• Review AI conversations involving real-world interpersonal scenarios.
• Identify missing context and suggest relevant follow-up questions.
• Evaluate whether extra information would change AI recommendations.
• Distinguish critical vs non-essential information for reviews.
• Conduct qualitative reviews and document findings using structured guidelines.
• Collaborate with research teams to ensure consistent assessments.

Skills

Analytical thinking

Attention to detail

Written communication

Objective evaluation

Data labeling tools

Spreadsheets

Education

MSW with LCSW or equivalent

MA/MS with LMFT/LPC/LCPC or equivalent

PhD or PsyD in psychology

MA/PhD in HCI, Sociology, Anthropology or related

5+ years advisory/consulting experience

Tools

Spreadsheets

Annotation platforms

Data labeling tools

Job description

Verita AI is seeking subject matter experts to review AI conversations and assess whether sufficient context was gathered before recommendations were made. You will identify missed opportunities for clarification and provide structured feedback to improve AI systems.

The role focuses on rigorous qualitative evaluation and documentation. Successful candidates will have licensure or advanced degrees in psychology, social work, or related fields, plus 5+ years of advisory or consulting experience.