Remote AI Evaluation Specialist – Critical Reasoning

mercor; United States; À distance; USD 80 000 - 113 000; Plein temps

Résumé du poste

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, this remote contract position seeks a Generalist Expert to evaluate AI-generated responses, strengthening reasoning and rigor in model outputs.

Responsibilities include providing structured written feedback, improving training data quality and downstream performance, and developing clear, well-evidenced rationales.

Qualifications

• Bachelor's degree from a top-500 globally ranked university preferred.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.
• Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
• Ability to write clear, precise, and well-evidenced written rationales.
• Consistent and honest judgment, including the ability to give critical assessments.
• Strict attention to detail and accurate application of structured evaluation guidelines.
• Ability to work entirely without AI writing tools.
• Native English fluency required.

Responsabilités

• Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs.
• Provide structured written feedback to improve training data quality and downstream performance.
• Develop clear, precise, and well-evidenced written rationales beyond surface-level observations.
• Apply strict attention to detail and accurate application of structured evaluation guidelines.
• Work independently and asynchronously to meet deadlines while improving AI model performance.

Connaissances

Analytical skills

Written communication

Independent work

Attention to detail

Critical reading

English fluency

Judgment

Ethical reasoning

Formation

Bachelor's degree (top-500 university)

Description du poste

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, this remote contract position seeks a Generalist Expert to evaluate AI-generated responses, strengthening reasoning and rigor in model outputs.

Responsibilities include providing structured written feedback, improving training data quality and downstream performance, and developing clear, well-evidenced rationales.