Manager, Assessment Development & Innovation

The Institute of Internal Auditors; Orlando (FL); Hybrid; USD 110,000 - 140,000; Full time

Job summary

The Institute of Internal Auditors (IIA) seeks a Manager of Assessment Development & Innovation to advance certification programs through rigorous test development practices. The role focuses on job task analysis, content creation, item review, and standard-setting to ensure exams are valid and defensible.

The candidate will also spearhead initiatives exploring new assessment methods and AI-powered item generation.

Qualifications

• Bachelor's degree in Psychology, Measurement, Education, or related field, or equivalent experience.
• 3–5+ years of assessment/test development, certification, or credentialing programs.
• Experience with job task analysis, item writing/review, and standard-setting methodologies.
• Strong facilitation skills guiding SME panels toward consensus.
• Excellent written communication for technical reports; proficient in survey design and data analysis using Excel.

Responsibilities

• Lead SME and stakeholder sessions throughout exam development lifecycle (JTA data gathering, item review, standard-setting).
• Develop interview protocols and conduct focus groups for JTA data collection.
• Design and implement JTA surveys to gather practitioner input at scale.
• Create PowerPoint materials and pre-meeting resources to support SME understanding of JTA and item review.
• Prepare data for standard-setting activities, including item-level statistics and rating materials.
• Manage end-to-end SME item curation, tracking, and quality control.
• Evaluate AI-generated exam items for accuracy, relevance, and alignment with content specs.
• Analyze survey and standard-setting data to validate tasks and knowledge statements.

Skills

Facilitation

Analytical thinking

Project management

Technical writing

Education

Bachelor's degree in Psychology, Measurement, Education, or related

Advanced degree preferred

Tools

Excel

Job description

About The IIA

The Institute of Internal Auditors (IIA) is the global leader and recognized authority for the internal audit profession. We advance the profession worldwide through education, advocacy, certifications, and professional resources. Guided by our values of People First, Integrity, Collaboration, Accountability, and Innovation, we offer a collaborative, purpose-driven workplace where employees can grow professionally while making a global impact.

Location

Lake Mary, FL (Hybrid)

Department

Global Certifications

Status

Full-Time | Exempt

Why Join The IIA?

• Make a global impact by helping ensure professional certification exams remain rigorous, relevant, and trusted.
• Help shape the future of assessment through innovative testing methods and emerging technologies.
• Collaborate with subject matter experts, volunteer committees, and teams across a global professional organization.
• Explore new approaches to assessment, including AI-assisted item generation and scoring.
• Grow your career in a collaborative, values-driven organization committed to innovation and continuous improvement.

Position Summary

The Institute of Internal Auditors (IIA) is seeking a Manager, Assessment Development & Innovation to support and advance our certification exam programs through rigorous, standards-aligned test development practices.

This role plays a central part in job task analysis (JTA), content development, item review, and standard-setting activities that help ensure certification exams remain valid, defensible, and reflective of current practice in the field.

The Manager will also lead independent initiatives exploring new assessment methods, innovative content development, and emerging technologies—including AI-assisted item generation and scoring—to strengthen and modernize The IIA's portfolio of certification programs.

Success in this position requires strong project management skills, sound psychometric judgment, analytical capabilities, and the ability to effectively facilitate and collaborate with subject matter experts (SMEs), volunteer committees, external stakeholders, and cross-functional internal teams.

Work Arrangement

This is a hybrid position based in Lake Mary, Florida.

What You'll Do

• Facilitate SME and stakeholder sessions throughout the exam development lifecycle, including JTA data-gathering sessions, item review panels, and standard-setting sessions using Modified Angoff or comparable methodologies.
• Develop interview protocols and conduct focus groups to support JTA data collection.
• Design and implement JTA surveys to gather practitioner input at scale.
• Develop PowerPoint presentations, pre-meeting materials, and other resources that support SME understanding of JTA, item curation and review, and standard-setting activities.
• Prepare and organize data for standard-setting activities, including item-level statistics and rating materials.
• Manage end-to-end SME item curation, including assignment, tracking, and quality control.
• Evaluate AI-generated exam items for accuracy, relevance, quality, and alignment with content specifications.
• Analyze survey and standard-setting data to validate task and knowledge statements, inform syllabus development, and confirm consistency and defensibility.
• Develop clear, accurate reports summarizing JTA methodology, findings, and recommendations for internal and external stakeholders.
• Write standard-setting reports suitable for accreditation and legal defensibility, including alignment with ISO/IEC 17024 and applicable Uniform Guidelines.
• Lead new assessment initiatives with minimal oversight, exercising sound judgment and proactive project management.
• Collaborate on or lead market research to benchmark exam development practices, emerging credentialing trends, and competitor programs.
• Design and test prototypes for new exam item types and formats.
• Explore and evaluate AI-based scoring approaches for open-response and constructed-response items, assessing feasibility, validity, and fairness implications.
• Collaborate closely with SMEs, volunteer committees, and external stakeholders, including chapters, Institutes, and Education programs.
• Partner with internal Certification, Education, Marketing, and other cross-functional teams.
• Coordinate, as needed, with external vendors supporting AI tools, testing platforms, or standard-setting logistics.
• Manage multiple concurrent projects and competing deadlines while maintaining quality, accuracy, and alignment with program objectives.

Required Qualifications

• Bachelor's degree in Psychology, Measurement, Education, or a related field, or equivalent professional experience.
• 3–5+ years of experience in assessment/test development, certification, or credentialing programs.
• Demonstrated experience with job task analysis, item writing and review, and standard-setting methodologies.
• Strong facilitation skills with the ability to effectively guide SME panels toward consensus.
• Excellent written communication skills, including experience producing clear technical reports for varied audiences.
• Proficiency with survey design and data analysis tools, including Excel.
• Strong analytical and data interpretation skills.
• Strong project management and organizational skills with the ability to manage multiple concurrent projects and competing deadlines.
• Ability to work independently, exercise sound judgment, and lead initiatives with minimal oversight.
• Ability to work effectively with SMEs, volunteer committees, external stakeholders, and cross-functional teams.
• Curiosity, adaptability, and willingness to explore emerging assessment technologies, including AI-assisted item generation and automated scoring.

Preferred Qualifications

• Advanced degree in Psychology, Psychometrics, Measurement, or a related field.
• Familiarity with ISO/IEC 17024 and other accreditation standards relevant to certification bodies.
• Experience evaluating or working with AI-generated content in a testing or content development environment.
• Exposure to AI or automated scoring approaches for constructed-response items.
• Experience with item banking or exam authoring platforms.
• Experience with statistical software or qualitative analysis tools.

Core Competencies

• Psychometric and measurement judgment
• Project management and self-direction
• SME and stakeholder facilitation
• Analytical and data interpretation skills
• Clear technical writing
• Curiosity and adaptability toward emerging testing technologies, including AI-assisted item generation and automated scoring

Equal Opportunity Employer

The Institute of Internal Auditors (IIA) is an Equal Opportunity Employer. We are committed to creating an inclusive workplace where all employees are treated with dignity and respect. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.