Test Construction Analyst

NBME; Philadelphia (Philadelphia County); Remote; USD 75,000 - 100,000; Full time

Job summary

NBME is offering a unique opportunity for two Test Construction Analysts to join our Test Development team. In this remote role, you will engage in critical analysis, test construction, and support across high-impact exams like the USMLE. Candidates with 3-5 years of relevant experience using tools like R and Python are encouraged to apply.

Qualifications

• 3 to 5 years of programming experience.
• Knowledge of assessment and psychometrics.
• Exposure to Automated Test Assembly (ATA).

Responsibilities

• Perform analyses by extracting data using R.
• Execute automated assembly of test forms.
• Monitor implementation of new technology and systems.

Skills

Data analysis

Communication skills

Attention to detail

Problem-solving

Learning new technology

Research process knowledge

Education

Bachelor's Degree

Tools

R

Python

Excel

SQL

C#

CPLEX

Job description

POSITION OVERVIEW

NBME is excited to announce an opportunity for two Test Construction Analysts to join our dynamic Test Development team. In this project-based role, you'll be responsible for providing critical analysis, test construction, publication services, and system support across a range of high-impact examination programs. These include the United States Medical Licensing Examination (USMLE), North American Veterinary Licensing Examination (NAVLE), Medical School Services, and NBME Self-Assessment Services. This is a unique chance to contribute to the quality and integrity of exams that shape the future of healthcare and veterinary professionals. While this is a project role expected to last between 18 to 24 months, individuals hired into this position will be eligible for the same comprehensive benefits as regular NBME employees.

This role has been designated as primarily remote, meaning it requires little to no onsite attendance. While this role can be designated as remote, you are offered the flexibility to select if you prefer to work primarily remotely, hybrid, or onsite. We're open to considering candidates within 50 miles of our office location in Philadelphia, PA.

Diversity, Equity, and Inclusion Statement

At NBME , we continue to innovate and improve how we fulfill the evolving needs of the health care community. This commitment starts and ends with the people at NBME. By recruiting and empowering talented individuals from various disciplines and backgrounds, which includes professionals with diverse life experiences, abilities, and perspectives, NBME can take a well-informed, robust approach to advancing medical education and assessment for years to come. We also continue to focus on ensuring that our DEI work is impactful and ingrained in everything we do, including with our staff, workplace culture, products and services, the Philadelphia community and the broader medical education landscape.

RESPONSIBILITIES

• Perform analyses by extracting data from our databases and data warehouses using R to generate the results
• Communicate results to non-technical members of the organization
• Execute automated assembly of test forms
• Analyze the statistical and content characteristics of the new forms using R, Python, and Excel
• Create, update, and disseminate documentation for new and existing technological procedures and routines, including direct user support and training
• Collaborate with other units in the research, design, and testing of new technology and software.
• Monitor implementation of new technology, systems, and software

QUALIFICATIONS

Skills and Abilities

• Significant expertise with software applications supporting data analysis/warehousing (R, Python, C#, Excel, SQL, CPLEX)
• Excellent interpersonal and communication skills including the ability to "translate" technical information to non-technical individuals
• Strong attention to detail and problem-solving skills
• Ability to learn and apply new technology quickly
• Ability to set priorities consistent with unit goals
• Some knowledge of and experience with the scholarly research process

Experience

• 3 to 5 years of relevant with programming at the level of the business unit
• Previous exposure with or knowledge about assessment, Item Response Theory (IRT) and Psychometrics
• Knowledge of optimized automated process with complex data, like Automated Test Assembly (ATA)

Education

• Bachelor's Degree

About NBME:

About NBME

NBME is a not-for-profit organization that specializes in the creation of assessments and learning tools for physicians and health professionals. Our mission is to advance assessment of these professionals to achieve optimal care for all, supporting the development of a highly effective, diverse and compassionate health care workforce. Founded in 1915, NBME develops and manages the United States Medical Licensing Examination with the Federation of State Medical Boards. We offer a comprehensive portfolio of assessment products for every stage of the medical school journey and provide assessment services to various health profession organizations. We are dedicated to advancing innovative assessment approaches through research, collaboration and contributions to the medical education and assessment communities. In 2024, we expanded our assessment capabilities to include simulation through the acquisition of MedVR Education. Learn more on NBME's website.

NBME's Community Collaborations and Contributions

NBME believes that a key path to meeting our mission is collaborating with and supporting our communities. Our Community Collaborations and Contributions programs provide resources and recognition to medical educators and researchers advancing the field of assessment throughout their careers. In addition, we invest in fee assistance, scholarships and pathway programs to aid learners and help support increased representation in health care. Headquartered in Philadelphia, NBME also gives to local organizations that advance health equity and access and positively impact social determinants of health.

#LI-Remote

#Associate