Analytical Evaluator - AI Feedback

Mercor; New York (NY); On-site; USD 60,000 - 80,000; Full time

Job summary

Obsidian is seeking a detail-oriented evaluator to review AI-generated responses and provide structured written feedback. You will assess accuracy, identify gaps in reasoning, and document findings following our established evaluation guidelines.

Ideal candidates demonstrate native English fluency, strong analytical writing, and the ability to work independently without AI writing tools. This role supports high-impact AI research initiatives and requires careful, precise judgment.

Qualifications

• Bachelor's degree from a top-500 globally ranked university preferred.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.

Skills

Critical reading

Clear written reasoning

Honest judgment

Attention to detail

Work without AI tools

Education

Bachelor's degree

Job description

Overview

In this role, you will evaluate AI-generated responses and provide structured written feedback. This is a great opportunity for sharp, analytical thinkers to contribute to high-impact AI research projects.

Basic Qualifications

• Bachelor's degree from a top-500 globally ranked university preferred
• Strong analytical and written communication skills
• Ability to work independently and follow detailed task guidelines

Required Skills

• Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning
• Ability to write clear, precise, and well-evidenced written rationales that go beyond surface-level observations
• Consistent and honest judgment, including the ability to give critical assessments when warranted
• Strict attention to detail and accurate application of structured evaluation guidelinesAbility to work entirely without AI writing tools

Eligibility

• Native English fluency required