AI Rater Guidelines Architect — Remote

mercor; United States; Remote; USD 62,000 - 90,000; Part time

Job summary

Mercor is seeking an AI Rater Guidelines Writer to translate complex program specs into precise, rater-ready instructions. You will design clear, non-contradictory rating guidelines across finance, retail, insurance, legal, and sports domains, ensuring consistency and accuracy in all guideline sets.

As a contractor, you will collaborate with program leads and subject matter experts, engaging reliably for at least 35 hours per week during weekdays, and you will shape guidance for GenAI/RLHF

Qualifications

• 3+ years of professional experience in linguistics, instructional design, technical writing, or a related field.
• Experience writing or refining guidelines/rubrics for human raters in a GenAI/RLHF context.
• Ability to work across multiple domains (finance, retail, insurance, legal, sports) and translate domain-specific nuances.
• Strong track record of resolving ambiguity and contradiction in written specifications.
• Ability to engage reliably for at least 35 hours/week during weekdays.
• Strong written communication skills for explaining complex guidance clearly.

Responsibilities

• Guide research and program teams to close the gap between ambiguous program requirements and rater-ready instructions across various domains.
• Design clear, non-contradictory rating guidelines and rubrics for consistent application by raters, including in edge cases.
• Evaluate draft guidelines for ambiguity, contradictions, and coverage gaps, revising until raters can apply them without escalation.
• Translate program specifications from domains such as finance, retail, insurance, legal, and sports into precise rater instructions.
• Collaborate with subject matter experts and program leads to ensure consistency and accuracy across guideline sets.

Skills

Linguistics

Instructional design

Technical writing

Guideline development

Ambiguity resolution

Domain translation

Written communication

Job description

Mercor is seeking an AI Rater Guidelines Writer to translate complex program specs into precise, rater-ready instructions. You will design clear, non-contradictory rating guidelines across finance, retail, insurance, legal, and sports domains, ensuring consistency and accuracy in all guideline sets.

As a contractor, you will collaborate with program leads and subject matter experts, engaging reliably for at least 35 hours per week during weekdays, and you will shape guidance for GenAI/RLHF