Remote | Educator/ Assessor Writer — $90–$140/hour

24-Mag Llc; Northern, New York (KY, NY); Hybrid; USD 124,000 - 193,000; Full time

Job summary

24-MAG LLC seeks experienced Educator/ Assessor Writers for remote evaluation of AI-generated written responses. You will apply structured rubrics, assess clarity and relevance, and provide detailed rationales to ensure consistent quality judgments across tasks.

Ideal candidates have a strong background in assessment, instructional design, and academic writing, with exceptional attention to detail and the ability to work independently in a remote setting.

Qualifications

• First-language proficiency in written English or equivalent.
• Experience marking, grading, examining, or reviewing written work against defined criteria.
• Background in assessment design, instructional design, peer review, thesis supervision, or academic writing support.

Responsibilities

• Evaluate AI-generated written responses for clarity, tone, usefulness, relevance, and instruction adherence.
• Apply rubric-based assessment and scoring to determine analytical quality scores.
• Identify factual errors, inconsistencies, and areas needing stronger support or justification.

Skills

English fluency

Instructional design

Academic writing

Critical analysis

Attention to detail

Remote collaboration

Education

US/CA/UK/AU degree

Tools

Digital content-review platforms

Job description

We are sharing a specialised consulting opportunity for experienced Educator/ Assessor Writers with strong expertise in assessment, grading, rubric-based evaluation, academic writing, instructional design, and critical content review to contribute to an advanced AI training and written-response evaluation project.

Selected professionals will evaluate AI-generated written responses, apply structured assessment criteria, identify factual and reasoning issues, and produce clear written rationales supporting consistent quality judgements. The opportunity is fully remote and suitable for experienced educators, assessors, academic reviewers, and instructional professionals. No prior experience in AI is required.

Key Responsibilities

Written Response Evaluation

• Evaluate and compare multiple AI-generated written responses
• Assess clarity, tone, helpfulness, relevance, and adherence to instructions
• Identify strengths and weaknesses across competing responses
• Evaluate whether content is appropriate for its intended audience
• Apply consistent professional judgement across repeated review tasks

Rubric‑Based Assessment & Scoring

• Apply provided rubrics and evaluation guidelines to written content
• Assign analytical quality scores using defined assessment criteria
• Distinguish between fluent, well-written responses and genuinely high-quality outputs
• Identify responses that are comprehensive but impractical or insufficiently supported
• Maintain consistency and objectivity across scoring decisions

Critical Analysis & Content Review

• Detect factual errors, unsupported claims, and internal inconsistencies
• Identify weaknesses affecting reliability, usability, or instructional value
• Assess reasoning quality and whether conclusions are adequately supported
• Review written material with strong attention to detail and analytical precision
• Flag ambiguous or borderline cases requiring additional consideration

Written Rationale & Documentation

• Produce detailed written rationales explaining assessment decisions
• Clearly justify scores, rankings, and evaluation outcomes
• Document findings, observations, and recurring quality issues
• Maintain thorough records of review decisions
• Communicate complex assessment judgements in concise, professional language

Assessment Design & Remote Collaboration

• Apply experience from grading, marking, instructional design, or academic review
• Contribute insight on rubric interpretation and effective assessment practices
• Collaborate asynchronously with project managers and reviewers
• Use digital content-review platforms to complete and document assignments
• Work independently while meeting project deadlines and quality expectations

Ideal Profile

• First-language proficiency or fully equivalent fluency in written English
• Completed degree from an accredited institution in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand
• Experience marking, grading, examining, or reviewing written work against defined criteria
• Background in assessment design, instructional design, peer review, thesis supervision, or academic writing support
• Strong critical-analysis and content‑evaluation skills
• Excellent written communication and editing ability
• Exceptional attention to detail and consistency
• Ability to make objective, well‑articulated assessment judgements
• Comfortable interpreting and applying structured rubrics
• Able to identify factual, logical, and usability issues in written content
• Comfortable working independently in a remote environment
• Familiarity with digital content‑review platforms is advantageous
• Experience with model evaluation, RLHF, annotation, or similar workflows is beneficial but not required

Engagement Details

• Independent contractor engagement
• Fully remote
• Compensation: $90–$140/hour
• Work will involve written‑response evaluation, rubric‑based scoring, critical analysis, editing, and detailed assessment rationale
• The engagement is designed to offer flexibility alongside other professional obligations
• Assignments may involve comparing multiple AI‑generated responses across varied topics and audiences
• Strong assessment judgement, written communication, and attention to detail are central to this engagement
• Collaboration will primarily be asynchronous through digital review and project‑management workflows
• Project scope, workload, evaluation criteria, and review standards may evolve depending on project requirements
• Work must be completed without using confidential or proprietary information belonging to any employer, client, institution, or other third party

About the Platform

This opportunity is available through 24‑MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project‑based workstreams.

By submitting this application, you acknowledge that your information may be processed by 24‑MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy