Applied Philosophy Benchmark Specialist

Mercor; United States; Remote; USD 69,000 - 87,000; Full time

Job summary

Mercor is seeking expert philosophers to author and review high-quality academic assessment content for an AI research initiative. You will write and verify rigorous multiple-choice questions across core philosophy domains, evaluate solution quality, and help establish gold-standard benchmarks used to advance AI capabilities.

You will be assigned one of two task types: Question Authoring or Question Verification, and you will work fully remotely with an expected commitment of 10+ hours per week.

Qualifications

• PhD or doctoral candidate in Philosophy or closely related field.
• Strong command of philosophical argumentation and canonical texts.

Responsibilities

• Author original philosophy questions testing deep conceptual understanding.
• Rate difficulty: Medium, Hard, or Expert for each item.
• Provide 1 correct answer and 9 plausible alternatives.
• Write step-by-step Chain-of-Thought solutions with reasoning (markdown).
• Supply 1–5 academic references per question.
• Verify and edit questions for clarity, completeness, precision, or solvability.

Skills

Academic writing

Formal logic

Critical thinking

Education

PhD in Philosophy

Job description

Pay: $50 - $63/hour
Commitment: Hourly
Field: Humanities

Role Overview

We are seeking expert philosophers to author and review high-quality academic assessment content for an AI research initiative. You will write and verify rigorous multiple-choice questions across core philosophy domains, evaluate solution quality, and help establish gold-standard benchmarks used to advance AI capabilities.

You will be assigned one of two task types:

• Question Authoring - Create original, challenging multiple-choice questions in your area of philosophy expertise, rate their difficulty, and submit them for review.
• Question Verification - Review pre-written questions for accuracy, clarity, and rigor. Edit where needed, rate difficulty, and document any changes made.

Philosophy Domains Covered

Formal Ontology & Knowledge Representation, AI Ethics, Applied Epistemology, Philosophy of Technology & Robotics, Philosophy of Science.

Key Responsibilities

• Author original philosophy questions that test deep conceptual understanding, not surface-level recall
• Ensure questions are unambiguous, self-contained, and precisely defined - all necessary information must be in the problem statement
• Rate each question's difficulty: Medium (intro undergraduate), Hard (advanced undergraduate), or Expert (post-graduate and above)
• Provide 1 correct answer and 9 plausible but subtly incorrect alternatives that challenge expert-level solvers
• Write step-by-step Chain-of-Thought solutions with clear, concise reasoning in markdown format
• Supply 1-5 academic references per question from reputable sources (peer-reviewed journals, university repositories)
• For verification tasks: flag issues with clarity, completeness, precision, or solvability and justify any edits made

Ideal Qualifications

• PhD or doctoral candidate in Philosophy or a closely related field
• Master's degree considered for candidates with exceptional depth in a specific subdomain
• Strong command of philosophical argumentation, formal logic, and canonical texts across traditions
• Research publications or teaching experience in philosophy is a strong plus
• Excellent written English and ability to express complex ideas clearly and concisely

More About the Opportunity

• Expected commitment: 10+ hours/week
• Asynchronous, fully remote work