Senior AI Assessment Lead — Item Quality & Fairness (Remote)

Alpheva; United States; Remote; USD 262,000 - 331,000; Part time

Job summary

Alpheva is seeking a senior assessment expert to work on AI evaluation projects focused on item quality, rubrics, validity, fairness, and score interpretation. You will review model responses and worked examples for misaligned learning objectives, invalid assessment assumptions, and unsupported pedagogical claims.

This remote, project-based contract offers hourly pay of $190–$240. Scope and hours depend on matching, with payout methods to be clarified after first payment.

Qualifications

• 7+ years of substantive experience in educational assessment or psychometrics.
• Senior-level judgment in item quality, rubrics, validity, fairness, and score interpretation with substantial responsibility.
• Ability to write precise reference answers, assess competing approaches, and justify corrections with traceable evidence.

Responsibilities

• Write precise reference answers and scoring criteria grounded in item quality, rubrics, validity, fairness, and score interpretation.
• Explain consequential errors, evidence gaps, and the reasoning behind corrections.
• Review model responses and worked examples for misaligned objectives and unsupported claims.

Skills

Item quality

Rubrics

Validity

Fairness

Score interpretation

Evidence-based justification

Written communication

Job description

Alpheva is seeking a senior assessment expert to work on AI evaluation projects focused on item quality, rubrics, validity, fairness, and score interpretation. You will review model responses and worked examples for misaligned learning objectives, invalid assessment assumptions, and unsupported pedagogical claims.

This remote, project-based contract offers hourly pay of $190–$240. Scope and hours depend on matching, with payout methods to be clarified after first payment.