Remote AI Evaluation Specialist

mercor; San Francisco (CA); Hybrid; GBP 44,000 - 73,000; Full time

Job summary

Mercor, a remote-first talent platform, is seeking a Generalist Expert to evaluate AI-generated responses for accuracy and depth. You will review outputs, cite evidence, and provide structured feedback to improve model performance.

This contract role emphasizes independent, asynchronous work and meeting deadlines. The ideal candidate holds a top-500 global university degree, has strong analytical and written communication skills, and fluency in English.

Qualifications

• Bachelor’s degree from a top-500 globally ranked university.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.
• Strong critical reading skills.
• Native English fluency required.

Responsibilities

• Evaluate AI-generated responses to ensure accuracy and depth in reasoning.
• Provide structured written feedback to improve AI model outputs.
• Identify nuances, implicit meanings, and gaps in reasoning in AI-generated content.
• Write clear, precise, and well-evidenced rationales beyond surface-level observations.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Skills

Analytical skills

Written communication

Native English

Education

Bachelor’s degree (top-500 uni)

Job description

Mercor, a remote-first talent platform, is seeking a Generalist Expert to evaluate AI-generated responses for accuracy and depth. You will review outputs, cite evidence, and provide structured feedback to improve model performance.

This contract role emphasizes independent, asynchronous work and meeting deadlines. The ideal candidate holds a top-500 global university degree, has strong analytical and written communication skills, and fluency in English.