Analytical Evaluator - AI Feedback

mercor; San Francisco (CA); Hybrid; GBP 44,000 - 73,000; Full time

Job summary

Mercor, a remote-first talent platform, is seeking a Generalist Expert to evaluate AI-generated responses for accuracy and depth. You will review outputs, cite evidence, and provide structured feedback to improve model performance.

This contract role emphasizes independent, asynchronous work and meeting deadlines. The ideal candidate holds a top-500 global university degree, has strong analytical and written communication skills, and fluency in English.

Qualifications

• Bachelor’s degree from a top-500 globally ranked university.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.
• Strong critical reading skills.
• Native English fluency required.

Responsibilities

• Evaluate AI-generated responses to ensure accuracy and depth in reasoning.
• Provide structured written feedback to improve AI model outputs.
• Identify nuances, implicit meanings, and gaps in reasoning in AI-generated content.
• Write clear, precise, and well-evidenced rationales beyond surface-level observations.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Skills

Analytical skills

Written communication

Native English

Education

Bachelor’s degree (top-500 uni)

Job description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Generalist Expert

Type: Contract

Compensation: $70/hour

Location: Remote

Role Responsibilities

• Evaluate AI-generated responses to ensure accuracy and depth in reasoning.
• Provide structured written feedback to improve AI model outputs.
• Identify nuances, implicit meanings, and gaps in reasoning in AI-generated content.
• Write clear, precise, and well-evidenced rationales beyond surface-level observations.
• Work independently and asynchronously to meet deadlines and improve AI model performance.

Qualifications

Must-Have

• Bachelor's degree from a top-500 globally ranked university preferred.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.
• Strong critical reading skills.
• Native English fluency required.

Preferred

• None specified.