Live AI Conversation Evaluator — Fluent English & JSON

Lifted, an Upwork Company™; Florida City (FL); On-site; USD 21,000 - 34,000; Part time

Job summary

Lifted, an Upwork Company™ is seeking experts to participate in live, real-time conversational scenarios with AI systems. You will review transcripts, label conversations, and provide written justifications, focusing on language quality, customer experience, and JSON structure.

Ideal candidates have strong JSON fluency, high attention to detail, and excellent written communication. A 15-minute assessment is required to proceed with onboarding, and pay is not provided for the assessment.

Qualifications

• Strong spoken-language proficiency in evaluation language.
• JSON fluency required - reading, validating, troubleshooting.
• Good audio quality and no background noise.

Responsibilities

• Participate in live, real-time conversational scenarios with AI systems.
• Review and label conversation transcripts.
• Evaluate AI-generated responses using structured rubrics and provide written justifications.
• Assess responses for language quality, customer experience, technical accuracy, formatting, and JSON structure.
• Capture required session artifacts (recordings, Session IDs, tool-call/console logs).

Skills

CEFR C1/C2

JSON fluency

Audio setup

Tools

JSON

Job description

Lifted, an Upwork Company™ is seeking experts to participate in live, real-time conversational scenarios with AI systems. You will review transcripts, label conversations, and provide written justifications, focusing on language quality, customer experience, and JSON structure.

Ideal candidates have strong JSON fluency, high attention to detail, and excellent written communication. A 15-minute assessment is required to proceed with onboarding, and pay is not provided for the assessment.