AI Response Evaluation Specialist

Mercor; New York (NY); On-site; USD 60,000 - 80,000; Full time

Job summary

Obsidian is seeking a detail-oriented evaluator to review AI-generated responses and provide structured written feedback. You will assess accuracy, identify gaps in reasoning, and document findings following our established evaluation guidelines.

Ideal candidates demonstrate native English fluency, strong analytical writing, and the ability to work independently without AI writing tools. This role supports high-impact AI research initiatives and requires careful, precise judgment.

Qualifications

• Bachelor's degree from a top-500 globally ranked university preferred.
• Strong analytical and written communication skills.
• Ability to work independently and follow detailed task guidelines.

Skills

Critical reading

Clear written reasoning

Honest judgment

Attention to detail

Work without AI tools

Education

Bachelor's degree

Job description

Obsidian is seeking a detail-oriented evaluator to review AI-generated responses and provide structured written feedback. You will assess accuracy, identify gaps in reasoning, and document findings following our established evaluation guidelines.

Ideal candidates demonstrate native English fluency, strong analytical writing, and the ability to work independently without AI writing tools. This role supports high-impact AI research initiatives and requires careful, precise judgment.