Remote AI Rating & Guidelines Writer

Weekday 1; United States; Remote; USD 62,000 - 90,000; Full time

Benefits offered by this job

Fully remote engagement

Job summary

Weekday 1 is seeking experienced linguists, instructional designers, technical writers, and AI content specialists for a fully remote, full-time role focused on creating precise rating guidelines and reviewer instructions for AI training data across finance, retail, insurance, legal, and sports domains.

Join a cutting-edge AI initiative to improve training quality by developing clear evaluation rubrics, guidelines, and edge-case coverage while ensuring consistency and reliability for human

Qualifications

• Minimum 3 years of professional experience in Linguistics, Instructional Design, Technical Writing, AI Content Development, or a closely related field.
• Direct experience creating, refining, or maintaining evaluation guidelines, rating rubrics, or reviewer instructions within Generative AI, RLHF, human evaluation, or AI data annotation environments.
• Demonstrated ability to work across multiple subject areas and convert domain-specific knowledge into clear, structured documentation.
• Proven experience resolving ambiguity and improving written specifications with measurable before-and-after improvements.

Responsibilities

• Translate complex and ambiguous program requirements into clear, concise, and easy-to-follow rater guidelines.
• Design comprehensive evaluation rubrics and scoring frameworks that enable consistent decision-making across a variety of AI tasks.
• Review existing documentation to identify ambiguity, inconsistencies, contradictions, and coverage gaps, then revise guidelines to improve clarity and usability.
• Convert subject-matter requirements from domains such as finance, retail, insurance, legal, and sports into structured evaluation instructions that non-domain raters can apply confidently.
• Collaborate with research teams, program managers, and subject matter experts to maintain consistency across multiple guideline sets.
• Develop documentation that addresses edge cases, exceptions, and complex evaluation scenarios while minimizing reviewer escalation.

Skills

Linguistics

Instructional Design

Technical Writing

AI Content Dev

Job description

Weekday 1 is seeking experienced linguists, instructional designers, technical writers, and AI content specialists for a fully remote, full-time role focused on creating precise rating guidelines and reviewer instructions for AI training data across finance, retail, insurance, legal, and sports domains.

Join a cutting-edge AI initiative to improve training quality by developing clear evaluation rubrics, guidelines, and edge-case coverage while ensuring consistency and reliability for human