Assessment Innovation Manager

ADP; Town of Florida (NY); Hybrid; USD 85,000 - 125,000; Full time

Job summary

The Institute of Internal Auditors (IIA) seeks a Manager of Assessment Development & Innovation, based in Lake Mary, FL, with a hybrid work arrangement. You will guide JTA data gathering, item review, and standard-setting activities to keep certification exams rigorous, defensible, and current.

You will also pilot AI-assisted item generation and scoring while coordinating with SMEs and cross-functional teams.

Qualifications

• > Bachelor’s degree in Psychology, Measurement, Education, or related field, or equivalent experience.
• - 3–5+ years in assessment/test development, certification, or credentialing programs.
• - Experience with job task analysis, item writing/review, and standard-setting methodologies.
• - Strong facilitation for SME panels; clear technical writing.
• - Proficiency with survey design and data analysis tools, including Excel.

Responsibilities

• Facilitate SME and stakeholder sessions across the exam development lifecycle using established methodologies.
• Develop interview protocols and conduct focus groups for JTA data collection.
• Design and implement JTA surveys to gather practitioner input at scale.
• Prepare PowerPoint presentations and materials to support SME understanding of JTA, item curation, review, and standard-setting.
• Organize data for standard-setting activities, including item-level statistics and rating materials.
• Manage end-to-end SME item curation with tracking and quality control.
• Evaluate AI-generated exam items for accuracy and alignment with content specifications.
• Analyze survey and standard-setting data to validate task/knowledge statements and syllabus development.
• Develop reports summarizing JTA methodology, findings, and recommendations for stakeholders.
• Write standard-setting reports suitable for accreditation and legal defensibility, including ISO/IEC 17024 alignment.
• Lead new assessment initiatives with minimal oversight and strong project management.
• Collaborate on market research to benchmark exam development practices and trends.
• Design and test prototypes for new exam item types and formats.
• Explore AI-based scoring approaches for open-response items; assess feasibility and fairness.

Skills

Facilitation

Written communication

Survey design

Data interpretation

Project management

Independent work

AI in assessment

Education

Bachelor's degree in Psychology/Measurement/Education

Equivalent professional experience

Advanced degree in Psychology/Psychometrics/Measurement

Tools

Excel

Job description

The Institute of Internal Auditors (IIA) seeks a Manager of Assessment Development & Innovation, based in Lake Mary, FL, with a hybrid work arrangement. You will guide JTA data gathering, item review, and standard-setting activities to keep certification exams rigorous, defensible, and current.

You will also pilot AI-assisted item generation and scoring while coordinating with SMEs and cross-functional teams.