Ai Session Reviewer - English Expert

Mercor; Sydney; Remote; AUD 33,000 - 41,000; Part time

Job summary

Mercor is seeking an AI Session Annotator (English) to work remotely. You will review recorded sessions where users interact with an AI assistant via voice and camera, and assess alignment with visible video content.

You will score sessions on visual interpretation, accuracy, relevance, response timing, and the use of spatial language, capturing evidence in the intake form. Immediate start with high workload between 26 to 31 Aug.

Qualifications

• Native or native-level English, spoken and written.
• Clear, precise written English.
• An Android phone with a working camera and microphone.

Responsibilities

• Review recorded sessions where users interact with an AI assistant via voice and camera.
• Compare the assistant's responses to visible video content. Identify hallucinated or fabricated visual details.
• Score sessions on visual interpretation, accuracy, relevance, response timing, and spatial language use.
• Evaluate if the assistant appropriately warned users about health, safety, or financial risks.
• Write detailed comments supporting scores with concrete session evidence.
• Capture conversation turns in the intake form.

Skills

Native English

Attention to detail

Written English

Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: AI Session Annotator (English)
Type: Contract
Compensation: $24–$30/hour
Location: Remote

Role Responsibilities

• Review recorded sessions where users interact with an AI assistant via voice and camera.
• Compare the assistant's responses to visible video content. Identify hallucinated or fabricated visual details.
• Score sessions on visual interpretation, accuracy, relevance, response timing, and spatial language use.
• Evaluate if the assistant appropriately warned users about health, safety, or financial risks.
• Write detailed comments supporting scores with concrete session evidence.
• Capture conversation turns in the intake form.

Qualifications

Must-Have

• Native or native-level English, spoken and written.
• Clear, precise written English.
• An Android phone with a working camera and microphone.
• Attention to detail and consistent rubric application.

Preferred

• Experience in data annotation, linguistic QA, content moderation, or AI model evaluation.
• Training in linguistics, translation, or interpreting.
• Ability to recognize unnatural or non-native English.

Start Date

• Immediate; high volume of work expected between 26 to 31 Aug.