Remote AI Feedback & Quality Evaluator

mercor; San Francisco (CA); Hybrid; AUD 158,000 - 237,000; Full time

Job summary

mercor is seeking a contract evaluator for user/customer research and feedback synthesis, working remotely. You will evaluate AI-generated artifacts against domain rubrics and provide structured feedback to improve AI outputs.

Role requires independence to meet deadlines and collaboration with subject matter experts to ensure dataset quality. Fluent English and an advanced degree are preferred.

Qualifications

• 5+ years of experience in user/customer research and feedback synthesis.
• Native or professional fluency in English.
• Proficiency in Microsoft Office and Google Workspace, especially Slides.
• Advanced degree (Master's or higher) preferred.

Responsibilities

• Evaluate AI-generated artifacts against domain-specific quality rubrics.
• Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks.
• Provide clear, structured written feedback to improve AI model outputs.
• Work independently and asynchronously to meet deadlines and enhance AI model performance.
• Collaborate with subject matter experts to ensure consistency and quality across datasets.

Skills

User research

Feedback synthesis

English fluency

Education

Master's degree or higher

Tools

Microsoft Office

Google Workspace

Job description

mercor is seeking a contract evaluator for user/customer research and feedback synthesis, working remotely. You will evaluate AI-generated artifacts against domain rubrics and provide structured feedback to improve AI outputs.

Role requires independence to meet deadlines and collaboration with subject matter experts to ensure dataset quality. Fluent English and an advanced degree are preferred.