Senior Exam Development Specialist

AzCEP; East Lansing, Northern (MI, KY); Hybrid; USD 65,000 - 90,000; Full time

Job summary

AzCEP seeks a Test Developer to draft exams, oversee quality control, and ensure integrity across medical assessments. You will guide programs through development and production, maintaining high standards for content and compliance.

You will collaborate with SMEs, editors, and psychometricians to align forms with blueprints, manage item banks, and contribute to committee reviews and publications. Strong communication and project skills are essential.

Qualifications

• Four-year degree and three to five years of relevant work experience in testing or a related field.
• At least two years of program or project management experience is required. Five years is experience is preferred.
• Understanding of question writing and examination construction principles is required. Expertise in this area is preferred.
• Understanding of measurement principles.
• Ability to collect and analyze data to inform decisions.
• Ability to learn medical terminology.
• Proficiency with Microsoft 365 (e.g., Excel, PowerPoint, SharePoint, Word); advanced Excel skills are preferred.
• Strong oral and written communication skills.
• Strong problem-solving and critical-thinking skills, and the ability to exercise sound judgement and make decisions based on accurate and timely analysis.
• Must maintain a high level of professionalism and confidentiality about sensitive matters and all organizational information.
• Ability to collaborate with individuals and teams in pursuit of continuous quality improvement.
• Ability to use web-based item banking applications (e.g., ExamDeveloper), test delivery platforms, and project management software (e.g., Trello, Smartsheet, HubSpot).
• Ability to travel up to 15% percent of the time.

Responsibilities

• Guides multiple examination programs by developing, updating, and coordinating project schedules and assignments for SMEs in collaboration with staff editors.
• Maintains item banks for multiple examination programs using standardized processes, conducting gap analyses as needed to ensure complete coverage of all domains in the examination blueprints.
• Develops examination forms for assigned examination programs, ensuring they align with blueprints and selection criteria, working with psychometricians to establish parameters.
• Guides SMEs through stages of examination development, including form assembly and review, publication, key validation and post-exam reviews.
• Organizes and guides exam development committee meetings and related logistics, including setting agendas, establishing timelines, and coordinating participant involvement.
• Prepares and contributes materials for committee, task force, and Board review.
• Assists with onboarding and training of new SMEs, including developing and updating training materials.
• Fosters and maintains strong relationships with committee members and staff to achieve outcomes.
• Ensures quality by offering test development expertise and sharing best practices with staff and SMEs.
• Contributes to the creation and upkeep of standard policies and procedures related to test development activities.
• Consults with psychometricians to support scoring tasks and other test-related projects.
• Develops, updates, and maintains test development reports for internal and external stakeholders.
• Provides technical support for staff and SMEs on testing applications, including testing software updates.
• Participates in planning and implementation of the annual Item Writers Workshop.
• Staffs case development meetings and provides ongoing support to case development teams.
• Develops and maintains documentation of item/case development and exam development processes while identifying opportunities for process improvements.
• Collaborates in problem-solving by analyzing, documenting, and addressing issues as they arise.
• Represents ABEM in presentations to stakeholders and professional audiences.
• Staffs the certification examination administrations and additional meetings as assigned.
• Performs other duties as needed or assigned.

Skills

Project management

Data analysis

Communication skills

Confidentiality

Team collaboration

Problem-solving

Travel up to 15%

Education

Bachelor's degree in a related field

Tools

ExamDeveloper

Trello

Smartsheet

HubSpot

Microsoft Excel

Job description

AzCEP seeks a Test Developer to draft exams, oversee quality control, and ensure integrity across medical assessments. You will guide programs through development and production, maintaining high standards for content and compliance.

You will collaborate with SMEs, editors, and psychometricians to align forms with blueprints, manage item banks, and contribute to committee reviews and publications. Strong communication and project skills are essential.