Remote AI Session Annotator & Evaluator

mercor; United Kingdom; Remote; GBP 25,000 - 31,000; Full time

Job summary

Mercor seeks an AI Session Annotator (English) for remote work. The role involves reviewing sessions where users interact with an AI assistant via voice and camera, scoring based on visual interpretation, accuracy, and timing, and highlighting any health/safety warnings.

Applicants should be native-level English with clear writing, possess an Android phone with camera/mic, and have strong attention to detail and rubric consistency. Prior data annotation or linguistic QA experience is preferred.

Qualifications

• Native or native-level English, spoken and written.
• Clear, precise written English.
• An Android phone with a working camera and microphone.
• Attention to detail and consistent rubric application.
• Experience in data annotation, linguistic QA, content moderation, or AI model evaluation.
• Training in linguistics, translation, or interpreting.
• Ability to recognize unnatural or non-native English.

Responsibilities

• Review recorded sessions where users interact with an AI assistant via voice and camera.
• Compare the assistant's responses to visible video content; identify hallucinated or fabricated visuals.
• Score sessions on visual interpretation, accuracy, relevance, response timing, and spatial language use.
• Evaluate if the assistant appropriately warned users about health, safety, or financial risks.
• Write detailed comments supporting scores with concrete session evidence.
• Capture conversation turns in the intake form.

Job description

Mercor seeks an AI Session Annotator (English) for remote work. The role involves reviewing sessions where users interact with an AI assistant via voice and camera, scoring based on visual interpretation, accuracy, and timing, and highlighting any health/safety warnings.

Applicants should be native-level English with clear writing, possess an Android phone with camera/mic, and have strong attention to detail and rubric consistency. Prior data annotation or linguistic QA experience is preferred.