Manager, Assessment Development & Innovation

Socket.dev; Town of Florida (NY); Hybrid; USD 90,000 - 140,000; Full time

Job summary

The Institute of Internal Auditors (IIA) is seeking a Manager, Assessment Development & Innovation to advance certification exam programs through rigorous test development practices in a hybrid Lake Mary, Florida setting.

You will lead JTA, content development, item review, and standard-setting, while exploring new assessment methods and AI-assisted item generation to strengthen the portfolio and ensure defensibility.

Qualifications

• Bachelor's degree in Psychology, Measurement, Education, or related field.
• 3–5+ years in assessment/test development, certification, or credentialing.
• Experience with job task analysis, item writing/review, and standard-setting.
• Strong facilitation skills guiding SME panels.
• Excellent technical writing for reports.
• Proficiency with survey design and data analysis; Excel.

Responsibilities

• Facilitate SME and stakeholder sessions throughout exam development.
• Develop interview protocols and focus groups for JTA data collection.
• Design and implement JTA surveys to gather practitioner input.
• Prepare data for standard-setting activities and item curation.
• Evaluate AI-generated exam items for accuracy and relevance.
• Lead new assessment initiatives with minimal oversight.
• Coordinate with external vendors for AI tools and testing platforms.

Skills

Facilitation

Written communication

Analytical thinking

Project management

Independent work

SME collaboration

Education

Bachelor's degree

Tools

Excel

Job description

Manager, Assessment Development & Innovation

Location: Lake Mary, FL (Hybrid)
Department: Global Certifications
Status: Full-Time | Exempt

About The IIA

The Institute of Internal Auditors (IIA) is the global leader and recognized authority for the internal audit profession. We advance the profession worldwide through education, advocacy, certifications, and professional resources. Guided by our values of People First, Integrity, Collaboration, Accountability, and Innovation, we offer a collaborative, purpose-driven workplace where employees can grow professionally while making a global impact.

Why Join The IIA?

• Make a global impact by helping ensure professional certification exams remain rigorous, relevant, and trusted.
• Help shape the future of assessment through innovative testing methods and emerging technologies.
• Collaborate with subject matter experts, volunteer committees, and teams across a global professional organization.
• Explore new approaches to assessment, including AI-assisted item generation and scoring.
• Grow your career in a collaborative, values-driven organization committed to innovation and continuous improvement.

Position Summary

The Institute of Internal Auditors (IIA) is seeking aManager, Assessment Development & Innovation to support and advance our certification exam programs through rigorous, standards-aligned test development practices.

This role plays a central part in job task analysis (JTA), content development, item review, and standard-setting activities that help ensure certification exams remain valid, defensible, and reflective of current practice in the field.

The Manager will also lead independent initiatives exploring new assessment methods, innovative content development, and emerging technologies—including AI-assisted item generation and scoring—to strengthen and modernize The IIA's portfolio of certification programs.

Success in this position requires strong project management skills, sound psychometric judgment, analytical capabilities, and the ability to effectively facilitate and collaborate with subject matter experts (SMEs), volunteer committees, external stakeholders, and cross-functional internal teams.

Work Arrangement

This is ahybrid position based in Lake Mary, Florida.

What You'll Do

• Facilitate SME and stakeholder sessions throughout the exam development lifecycle, including JTA data-gathering sessions, item review panels, and standard-setting sessions using Modified Angoff or comparable methodologies.
• Develop interview protocols and conduct focus groups to support JTA data collection.
• Design and implement JTA surveys to gather practitioner input at scale.
• Develop PowerPoint presentations, pre-meeting materials, and other resources that support SME understanding of JTA, item curation and review, and standard-setting activities.
• Prepare and organize data for standard-setting activities, including item-level statistics and rating materials.
• Manage end-to-end SME item curation, including assignment, tracking, and quality control.
• Evaluate AI-generated exam items for accuracy, relevance, quality, and alignment with content specifications.
• Analyze survey and standard-setting data to validate task and knowledge statements, inform syllabus development, and confirm consistency and defensibility.
• Develop clear, accurate reports summarizing JTA methodology, findings, and recommendations for internal and external stakeholders.
• Write standard-setting reports suitable for accreditation and legal defensibility, including alignment with ISO/IEC 17024 and applicable Uniform Guidelines.
• Lead new assessment initiatives with minimal oversight, exercising sound judgment and proactive project management.
• Collaborate on or lead market research to benchmark exam development practices, emerging credentialing trends, and competitor programs.
• Design and test prototypes for new exam item types and formats.
• Explore and evaluate AI-based scoring approaches for open-response and constructed-response items, assessing feasibility, validity, and fairness implications.
• Collaborate closely with SMEs, volunteer committees, and external stakeholders, including chapters, Institutes, and Education programs.
• Partner with internal Certification, Education, Marketing, and other cross-functional teams.
• Coordinate, as needed, with external vendors supporting AI tools, testing platforms, or standard-setting logistics.
• Manage multiple concurrent projects and competing deadlines while maintaining quality, accuracy, and alignment with program objectives.

Qualifications

Required Qualifications

• Bachelor's degree in Psychology, Measurement, Education, or a related field, or equivalent professional experience.
• 3–5+ years of experience in assessment/test development, certification, or credentialing programs.
• Demonstrated experience with job task analysis, item writing and review, and standard-setting methodologies.
• Strong facilitation skills with the ability to effectively guide SME panels toward consensus.
• Excellent written communication skills, including experience producing clear technical reports for varied audiences.
• Proficiency with survey design and data analysis tools, including Excel.
• Strong analytical and data interpretation skills.
• Strong project management and organizational skills with the ability to manage multiple concurrent projects and competing deadlines.
• Ability to work independently, exercise sound judgment, and lead initiatives with minimal oversight.
• Ability to work effectively with SMEs, volunteer committees, external stakeholders, and cross-functional teams.
• Curiosity, adaptability, and willingness to explore emerging assessment technologies, including AI-assisted item generation and automated scoring.

Preferred Qualifications

• Advanced degree in Psychology, Psychometrics, Measurement, or a related field.
• Familiarity with ISO/IEC 17024 and other accreditation standards relevant to certification bodies.
• Experience evaluating or working with AI-generated content in a testing or content development environment.
• Exposure to AI or automated scoring approaches for constructed-response items.
• Experience with item banking or exam authoring platforms.
• Experience with statistical software or qualitative analysis tools.

Core Competencies

• Psychometric and measurement judgment
• Project management and self-direction
• SME and stakeholder facilitation
• Analytical and data interpretation skills
• Clear technical writing
• Curiosity and adaptability toward emerging testing technologies, including AI-assisted item generation and automated scoring

Equal Opportunity Employer

The Institute of Internal Auditors (IIA) is an Equal Opportunity Employer. We are committed to creating an inclusive workplace where all employees are treated with dignity and respect. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable federal, state, or local law.