Remote | Educator/ Assessor Writer — $90–$140/hour

24-MAG; United States; Remote; USD 124,000 - 193,000; Full time

Job summary

24-MAG LLC offers a fully remote, contractor-based opportunity for experienced educators and evaluators to assess AI-generated written responses using structured rubrics. You will deliver concise, well-justified evaluations and rationales while maintaining high accuracy and consistency across tasks.

The role emphasizes independent work, strong editing skills, and meticulous attention to detail, with flexible scheduling and remote collaboration to fit other commitments.

Qualifications

• First-language proficiency in written English.
• Experience marking, grading, examining, or reviewing written work against defined criteria.
• Background in assessment design, instructional design, peer review, or academic writing support.
• Strong critical-analysis and content-evaluation skills.
• Excellent written communication and editing ability.
• Ability to work independently in a remote environment.
• Familiarity with digital content-review platforms is advantageous.
• Experience with model evaluation, RLHF, annotation, or similar workflows is beneficial but not required.

Responsibilities

• Evaluate AI-generated written responses and apply structured assessment criteria.
• Identify factual and reasoning issues and produce clear written rationales.
• Apply rubrics and evaluation guidelines to scoring and rankings.
• Detect factual errors, unsupported claims, and inconsistencies.
• Produce detailed written rationales explaining assessment decisions.
• Collaborate asynchronously with project managers and reviewers.

Skills

English fluency

Assessment design

Written evaluation

Attention to detail

Remote collaboration

Education

Bachelor's degree or higher

Tools

Content-review platforms

Job description

We are sharing a specialised consulting opportunity for experienced Educator/ Assessor Writers with strong expertise in assessment, grading, rubric-based evaluation, academic writing, instructional design, and critical content review to contribute to an advanced AI training and written-response evaluation project.

Selected professionals will evaluate AI-generated written responses, apply structured assessment criteria, identify factual and reasoning issues, and produce clear written rationales supporting consistent quality judgements. The opportunity is fully remote and suitable for experienced educators, assessors, academic reviewers, and instructional professionals. No prior experience in AI is required.

Key Responsibilities

Written Response Evaluation

• Evaluate and compare multiple AI-generated written responses
• Assess clarity, tone, helpfulness, relevance, and adherence to instructions
• Identify strengths and weaknesses across competing responses
• Evaluate whether content is appropriate for its intended audience
• Apply consistent professional judgement across repeated review tasks

Rubric-Based Assessment & Scoring

• Apply provided rubrics and evaluation guidelines to written content
• Assign analytical quality scores using defined assessment criteria
• Distinguish between fluent, well-written responses and genuinely high-quality outputs
• Identify responses that are comprehensive but impractical or insufficiently supported
• Maintain consistency and objectivity across scoring decisions

Critical Analysis & Content Review

• Detect factual errors, unsupported claims, and internal inconsistencies
• Identify weaknesses affecting reliability, usability, or instructional value
• Assess reasoning quality and whether conclusions are adequately supported
• Review written material with strong attention to detail and analytical precision
• Flag ambiguous or borderline cases requiring additional consideration

Written Rationale & Documentation

• Produce detailed written rationales explaining assessment decisions
• Clearly justify scores, rankings, and evaluation outcomes
• Document findings, observations, and recurring quality issues
• Maintain thorough records of review decisions
• Communicate complex assessment judgements in concise, professional language

Assessment Design & Remote Collaboration

• Apply experience from grading, marking, instructional design, or academic review
• Contribute insight on rubric interpretation and effective assessment practices
• Collaborate asynchronously with project managers and reviewers
• Use digital content-review platforms to complete and document assignments
• Work independently while meeting project deadlines and quality expectations

Ideal Profile

• First-language proficiency or fully equivalent fluency in written English
• Completed degree from an accredited institution in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand
• Experience marking, grading, examining, or reviewing written work against defined criteria
• Background in assessment design, instructional design, peer review, thesis supervision, or academic writing support
• Strong critical-analysis and content-evaluation skills
• Excellent written communication and editing ability
• Exceptional attention to detail and consistency
• Ability to make objective, well-articulated assessment judgements
• Comfortable interpreting and applying structured rubrics
• Able to identify factual, logical, and usability issues in written content
• Comfortable working independently in a remote environment
• Familiarity with digital content-review platforms is advantageous
• Experience with model evaluation, RLHF, annotation, or similar workflows is beneficial but not required

Engagement Details

• Independent contractor engagement
• Fully remote
• Compensation: $90--$140/hour
• Work will involve written-response evaluation, rubric-based scoring, critical analysis, editing, and detailed assessment rationale
• The engagement is designed to offer flexibility alongside other professional obligations
• Assignments may involve comparing multiple AI-generated responses across varied topics and audiences
• Strong assessment judgement, written communication, and attention to detail are central to this engagement
• Collaboration will primarily be asynchronous through digital review and project-management workflows
• Project scope, workload, evaluation criteria, and review standards may evolve depending on project requirements
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party

About the Platform

This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams.