JSON Evaluator - Fully Remote | Upto $175/hr

mercor; United States; Remote; USD 69,000 - 241,000; Part time

Job summary

Mercor is seeking a JSON-fluent & Rubrics Expert for a remote, US-based contract role offering 20–40 hours per week. Compensation is hourly.

The role involves evaluating JSON outputs, reading task descriptions and rubrics, and grading results against defined criteria with strict attention to detail. Start today with rapid turnaround expectations and independent work.

Qualifications

• Ability to read and interpret JSON files accurately.
• Experience evaluating outputs against defined rubrics is a plus.
• Based in the United States with availability today through July 27-28.

Responsibilities

• Evaluate JSON-formatted AI model outputs against defined grading rubrics.
• Read and understand written task descriptions and rubrics.
• Parse, read, and assess JSON-based outputs, grading them against provided rubrics.
• Ensure attention to detail and consistency in applying rubric criteria.
• Work independently to meet tight deadlines within 12-24 hours.

Skills

JSON

Rubrics evaluation

Attention to detail

Job description

About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: JSON-fluent & Rubrics Expert

Type: Contract

Compensation: $50–$175/hour

Location: Remote

Commitment: 20–40 hours/week

Role Responsibilities

• Evaluate JSON-formatted AI model outputs against defined grading rubrics.
• Read and understand written task descriptions and rubrics.
• Parse, read, and assess JSON-based outputs, grading them against provided rubrics.
• Ensure attention to detail and consistency in applying rubric criteria.
• Work independently to meet tight deadlines within 12-24 hours.

Qualifications Must-Have

• Available to start today, July 24 , running to July 27-28.
• Comfortable reading and interpreting JSON files.
• Based in the US.
• Access to a desktop or laptop computer (no Chromebooks).

Qualifications Preferred

• Prior Mercor experience, especially in applying detailed rubrics and evaluation criteria consistently.

Start Date

• Today, July 24

Resources & Support

• For details about the interview process and platform information, please check:
• For any help or support, reach out to: