Remote AI Session Evaluator (English) - Contract

mercor; San Francisco (CA); Hybrid; CAD 47,000 - 59,000; Part time

Job summary

Mercor is seeking an AI Session Annotator (English) for a remote, contract role. You will review recorded sessions where users interact with an AI assistant via voice and video, compare responses to visible content, and score sessions on visual interpretation, accuracy, relevance, response timing, and use of spatial language.

The ideal candidate is a native or near-native English speaker with clear, precise writing, strong attention to detail, and prior experience in data annotation or AI model

Qualifications

• Native or near-native English, spoken & written.
• Clear, precise written English.
• An Android phone with a working camera and microphone.
• Attention to detail and consistent rubric application.

Responsibilities

• Review recorded sessions where users interact with an AI assistant via voice and camera.
• Compare the assistant's responses to visible video content.
• Score sessions on visual interpretation, accuracy, relevance, response timing, and spatial language use.
• Evaluate if the assistant appropriately warned users about health, safety, or financial risks.
• Write detailed comments supporting scores with concrete session evidence.
• Capture conversation turns in the intake form.

Skills

Native English speaker

Clear written English

Attention to detail

Tools

Android phone with camera and microphone

Job description

Mercor is seeking an AI Session Annotator (English) for a remote, contract role. You will review recorded sessions where users interact with an AI assistant via voice and video, compare responses to visible content, and score sessions on visual interpretation, accuracy, relevance, response timing, and use of spatial language.

The ideal candidate is a native or near-native English speaker with clear, precise writing, strong attention to detail, and prior experience in data annotation or AI model