Lead Assessment Designer (US - Contract)

LearningMate; United States; Sur place; USD 110 000 - 160 000; Plein temps

Résumé du poste

LearningMate is seeking an experienced assessment designer to lead a pod of contract Assessment Developers creating Verified Skill Assessments. You will turn approved skill blueprints into assessment designs, guide daily work, and ensure each unit meets standards before reviewer handoff.

Responsibilities include designing synthesis tasks, leading a distributed team, and coordinating with psychometricians to ensure equal difficulty across forms.

Qualifications

• 5+ years designing assessments, including performance-based tasks scored with rubrics.
• Experience leading or reviewing other assessment designers' or developers' work.
• Working knowledge of Evidence-Centered Design or a similar claim-to-evidence method.
• Able to read basic item statistics, such as difficulty and discrimination, and act on them.
• Uses AI tools to speed up drafting, and checks their output carefully.
• Clear writer, well organized, and comfortable documenting decisions for a distributed team.
• A plus: experience with certification or licensure programs, standard setting, or expertise in AI, data, business or IT and security.

Responsabilités

• Turn approved skill blueprints into assessment designs using Evidence-C-centered Design.
• Design the synthesis task for each VSA to combine subskills in one realistic piece of work.
• Lead a pod of Assessment Developers: assign work, set weekly goals, coach on quality, and track throughput.
• Review every unit before in-house review for alignment, rubric quality, realism, clarity, accessibility and fairness.
• Plan equivalent forms with guidance from psychometricians, ensuring consistent difficulty across tests.
• Run grader testing for the pod with answer keys, borderline responses, and common wrong answers at each rubric band.
• Ensure the pod follows the program's security rules.
• Revise units using field test results, item analysis and reviewer feedback.
• Escalate issues to the in-house assessment design lead.

Connaissances

Assessment design

Leadership

Evidence-Centered Design

Rubrics & scoring

AI drafting tools

Documentation

Description du poste

We're looking for an experienced assessment designer to lead a pod of contract Assessment Developers building Verified Skill Assessments. These are secure, performance-based assessments that issue a credential employers can rely on. You'll turn approved skill blueprints into assessment designs, guide the pod's daily work, and make sure every unit meets the standard before it reaches our in-house reviewers.

Engagement: full-time contract.

Key responsibilities

• Turn approved skill blueprints into assessment blueprints using Evidence-Centered Design: the claim, the evidence that supports it, the tasks, the rubric criteria and the form plan.
• Design the synthesis task for each VSA, which brings its subskills together in one realistic piece of work.
• Lead a pod of Assessment Developers: assign work, set weekly goals, coach on quality, and track throughput and rework.
• Review every unit before in-house review: alignment to the blueprint, rubric quality, realism, clarity, accessibility and fairness.
• Plan equivalent forms with guidance from our psychometricians, so every version of a test is equally hard.
• Run grader testing for the pod with answer keys, borderline responses and common wrong answers at each rubric band.
• Make sure the pod follows the program's security rules.
• Revise units using results from field testing, item analysis and reviewer feedback.
• Escalate issues to the in-house assessment design lead.

Qualifications

• 5+ years designing assessments, including performance-based tasks scored with rubrics.
• Experience leading or reviewing other assessment designers' or developers' work.
• Working knowledge of Evidence-Centered Design or a similar claim-to-evidence method.
• Able to read basic item statistics, such as difficulty and discrimination, and act on them.
• Uses AI tools to speed up drafting, and checks their output carefully.
• Clear writer, well organized, and comfortable documenting decisions for a distributed team.
• A plus: experience with certification or licensure programs, taking part in standard setting, or expertise in AI, data, business or IT and security.