Medical Assessments Test Developer

American Board of Emergency Medicine; East Lansing (MI); On-site; USD 65,000 - 92,000; Full time

Job summary

The American Board of Emergency Medicine is seeking a Test Developer to draft exams, manage item banks, and oversee quality control across exam programs. You will guide SMEs, coordinate schedules, and ensure alignment with blueprints and scoring parameters.

Responsibilities include developing forms, reviewing content with psychometricians, and delivering documentation and training materials while maintaining confidentiality and high professional standards.

Qualifications

• Four-year degree and three to five years of relevant work experience in testing or a related field.
• At least two years of program or project management experience.
• Understanding of question writing and examination construction principles.
• Understanding of measurement principles.

Responsibilities

• Guides multiple examination programs by developing, updating, and coordinating project schedules and assignments for SMEs in collaboration with staff editors.
• Maintains item banks for multiple examination programs using standardized processes, conducting gap analyses as needed to ensure complete coverage of all domains in the examination blueprints.
• Develops examination forms for assigned examination programs, ensuring they align with blueprints and selection criteria.
• Works with psychometricians to establish selection parameters and necessary linkages.
• Guides SMEs through various stages of examination development, including form assembly and review, publication, key validation and post-exam reviews.
• Organizes and guides exam development committee meetings and related logistics, including setting agendas, establishing timelines, and coordinating participant involvement, in collaboration with staff editors.
• Prepares and contributes materials for committee, task force, and Board review.
• Assists with the onboarding and training of new SMEs, including developing and updating training materials in collaboration with staff editors.
• Fosters and maintains strong relationships with committee members and staff to achieve desired outcomes.
• Ensures quality by offering test development expertise and sharing best practices with staff and SMEs.
• Contributes to the creation and upkeep of standard policies and procedures related to test development activities.
• Consults with psychometricians as needed to support scoring tasks and other test-related projects.
• Develops, updates, and maintains test development reports for both internal and external stakeholders.
• Provides technical support for staff and SMEs on testing applications, including testing software updates as needed.
• Participates in the planning and implementation of the annual Item Writers Workshop, including on-site support and management of SMEs.
• Staffs case development meetings and provides ongoing support to case development teams between meetings as assigned.
• Develops and maintains documentation of item/case development and exam development processes while identifying opportunities for process improvements.
• Collaborates in problem-solving by analyzing, documenting, and addressing issues as they arise.
• Represents ABEM in presentations to stakeholders and professional audiences (e.g., ATP, ICE, ABMS Conference).
• Staffs the certification examination administrations and additional meetings as assigned.
• Performs other duties as needed or assigned.

Skills

Program management

Data analysis

Microsoft 365

Excel

PowerPoint

SharePoint

Word

Communication

Collaboration

Web-based item banking

Education

Bachelor's degree

Tools

ExamDeveloper

Trello

Smartsheet

HubSpot

Job description

The American Board of Emergency Medicine is seeking a Test Developer to draft exams, manage item banks, and oversee quality control across exam programs. You will guide SMEs, coordinate schedules, and ensure alignment with blueprints and scoring parameters.

Responsibilities include developing forms, reviewing content with psychometricians, and delivering documentation and training materials while maintaining confidentiality and high professional standards.