Senior Director, Assessment

Physician-Assistant-Education-Association; United States; Remote; USD 147,000 - 158,000; Full time

Benefits offered by this job

Remote work

Job summary

PA Education Association (PAEA) seeks a Senior Director, Assessment to lead the assessment portfolio, guide strategy, and drive continuous improvement. You will oversee psychometrics, exam development, and data-driven innovations while managing a remote, national team.

The role requires an 8+ year background in assessment with PhD-level credentials, strong leadership, and excellent communication skills to collaborate with executives and external partners. Competitive salary and benefits offered.

Qualifications

• Doctoral degree in Psychometrics, Educational Measurement, Quantitative Psychology, Educational Psychology, Statistics, Research Methodology, or closely related field.
• Minimum of 8 years of progressively responsible experience in assessment, psychometrics, educational measurement, testing, or related fields.
• Minimum of 3 years of supervisory or leadership experience managing professional staff and complex projects.
• Deep knowledge of Classical Test Theory, Item Response Theory, Rasch measurement, standard setting, test development, validation, scoring, equating, and assessment reporting.
• Demonstrated experience leading assessment programs, initiatives, or portfolios.
• Experience managing external vendors, consultants, and stakeholder relationships.
• Strong strategic planning, organizational leadership, and project management skills.
• Excellent written and verbal communication skills, including the ability to explain technical concepts to non-technical audiences.
• Ability to manage multiple priorities and lead complex cross-functional initiatives.
• Strong analytical, problem-solving, and decision-making skills.
• Demonstrated commitment to assessment quality, fairness, accessibility, and continuous improvement.

Responsibilities

• Lead the strategy, planning, implementation, evaluation, and continuous improvement of current and future assessment products and services.
• Monitor developments in educational measurement, psychometrics, assessment technology, artificial intelligence, and health professions education to identify opportunities for improvement.
• Collaborate with organizational leaders on assessment initiatives, business planning, and product development opportunities.
• Oversee the full assessment lifecycle, including blueprint development and evaluation, item banking, test construction, administration and delivery, scoring, scaling, equating, standard setting, reporting, quality assurance, and technical documentation.
• Ensure assessments are valid, reliable, fair, accessible, secure, and legally defensible, and establish related policies and procedures.
• Lead psychometric analyses and measurement research to support evidence-based decision-making.
• Guide the development and use of data tools, analytical solutions, and measurement technologies, and communicate findings to technical and nontechnical audiences.
• Supervise, coach, and mentor Assessment staff; manage performance; and foster alignment across psychometric, operational, and strategic functions.
• Oversee vendor performance and external partnerships; implement process improvements for quality, efficiency, and stakeholder experience.

Skills

Strategic leadership

Project management

Communication skills

Analytical thinking

Team leadership

Education

Doctoral degree in Psychometrics / Educational Measurement

Tools

R

Data visualization tools

Assessment platforms

Job description

The PA Education Association (PAEA) is seeking a Senior Director, Assessment to lead, develop, and grow a high-performing assessment team while guiding the organization's assessment strategy. The ideal candidate is an experienced people leader with a proven record of building, mentoring, and expanding teams that deliver excellence, collaboration, and continuous improvement. This leader also brings a strong foundation in psychometrics and educational measurement and a commitment to innovation and mission-driven impact.

The Senior Director, Assessment provides strategic and operational leadership for PAEA's assessment portfolio, translating organizational goals into assessment strategy and operational plans and ensuring that high-quality assessment programs and services are developed, delivered, evaluated, and continuously improved. Working closely with the Chief Data Officer and Principal Psychometrician, this position oversees assessment operations, psychometric and measurement activities, exam development, innovation initiatives, and external partnerships and vendors, ensuring that PAEA assessments remain valid, reliable, fair, defensible, and aligned with organizational strategy. The Senior Director also supports the development of future assessment products and services and represents PAEA within the broader assessment, measurement, and health professions education communities.

This position may be exposed to sensitive and confidential information daily, this individual is responsible for maintaining the confidentiality that knowledge of such warrants. In this position, employees must respect and protect the privacy, confidentiality, and security of all personal information to which they have access in the performance of their duties for PAEA, in accordance with PAEA’s Privacy Policy and the PAEA Employee Handbook that all employees must sign.

Essential Functions:

Reasonable accommodations may be made to enable individuals with disabilities to perform these essential functions.

Assessment Strategy and Innovation

• Lead the strategy, planning, implementation, evaluation, and continuous improvement of current and future assessment products and services, aligning short- and long-term priorities with organizational goals and member needs.
• Monitor developments in educational measurement, psychometrics, assessment technology, artificial intelligence, and health professions education to identify opportunities to improve assessment design, delivery, reporting, and product development.
• Collaborate with organizational leaders on assessment initiatives, business planning, and product development opportunities.

Assessment Program and Measurement Oversight

• Oversee the full assessment lifecycle, including blueprint development and evaluation, item banking, test construction, administration and delivery, scoring, scaling, equating, standard setting, reporting, quality assurance, and technical documentation.
• Ensure assessments are valid, reliable, fair, accessible, secure, and legally defensible, and comply with testing standards, industry best practices, organizational policies, and governance requirements. Establish and maintain related assessment policies and procedures.
• Lead psychometric analyses and measurement research, including item and test performance monitoring, scoring methodology, standard setting, applied research, corrective action, and continuous evaluation, to support evidence-based decision-making.
• Guide the development and use of data tools, analytical solutions, and measurement technologies, and communicate technical findings, score interpretation guidance, and assessment information to technical and nontechnical audiences.

Team Leadership and Development

• Supervise, coach, and mentor Assessment staff; manage performance; and foster alignment across psychometric, operational, and strategic functions.
• Support staff development and participation in professional organizations, conferences, publications, and research, while strengthening organizational capacity through cross-training, knowledge sharing, and succession planning.

Operations and Partnerships

• Oversee exam development activities, production schedules, budgets, resource allocation, volunteer engagement, and vendor performance.
• Coordinate the work of assessment staff, subject matter experts, vendors, and external partners, and cultivate relationships that advance PAEA’s assessment strategy.
• Implement process improvements that enhance quality, efficiency, scalability, sustainability, and the stakeholder experience, and support assessment-related meetings, summits, workshops, and professional development activities.

External and Professional Leadership

• Represent PAEA through presentations, publications, workshops, consultation, and professional service within the testing, psychometrics, educational measurement, credentialing, and health professions education communities.
• Strengthen members’ assessment literacy through educational programming, consultation, score interpretation resources, and the dissemination of best practices.

Security and Compliance

• Protect the confidentiality, integrity, and security of assessment materials, data, and processes, and maintain related policies, procedures, and risk controls.
• Coordinate incident response, risk mitigation, and compliance activities with organizational leaders, legal counsel, and other relevant stakeholders.

Other Duties

The duties and responsibilities listed above are representative of the nature and level of work assigned and are not necessarily all-inclusive. Duties, responsibilities, and activities may change at any time with or without notice.

Qualifications, Knowledge, Skills, and Abilities:

Required

• Doctoral degree (PhD, EdD, or equivalent) in Psychometrics, Educational Measurement, Quantitative Psychology, Educational Psychology, Statistics, Research Methodology, or a closely related field.
• Minimum of 8 years of progressively responsible experience in assessment, psychometrics, educational measurement, testing, or related fields.
• Minimum of 3 years of supervisory or leadership experience managing professional staff and complex projects.
• Deep knowledge of Classical Test Theory, Item Response Theory, Rasch measurement, standard setting, test development, validation, scoring, equating, and assessment reporting.
• Demonstrated experience leading assessment programs, initiatives, or portfolios.
• Experience managing external vendors, consultants, and stakeholder relationships.
• Strong strategic planning, organizational leadership, and project management skills.
• Excellent written and verbal communication skills, including the ability to explain technical concepts to non-technical audiences.
• Ability to manage multiple priorities and lead complex cross-functional initiatives.
• Strong analytical, problem-solving, and decision-making skills.
• Demonstrated commitment to assessment quality, fairness, accessibility, and continuous improvement.

Preferred:

• Experience in health professions education and assessment.
• Experience developing innovative assessment products, technologies, or services.
• Record of professional presentations, publications, or contributions to the assessment and measurement field.
• Experience managing assessment security programs and policies.
• Experience with R, data visualization tools, assessment platforms, and modern analytical technologies.
• Active involvement in professional measurement or testing organizations.

ADA Specifications

Must be able to perform essential duties satisfactorily with reasonable accommodation.

Travel Requirements

Periodic travel will be required for this role to all-staff meetings, PAEA events, professional development opportunities, and any other meetings as deemed necessary.

This position is remote, with no in-office presence required. PAEA offers a competitive salary and benefits that considers external market compensation and maintains internal equity. The salary range for this position is $147,000-$158,000 annually. The benefits package includes health, life, and disability insurances; paid time off and holidays; a 403(b) retirement plan with an 8% employer contribution; monthly wellness stipend, professional development funding, tuition assistance , and other benefits.

Work Authorization or Security Clearance Requirements

The individual in this position must be authorized to work in the United States.

Fairness and Accessibility

PAEA is committed to fairness and accessibility in all areas of its business activities, including employment, management, procurement, and contracting. As such, it is the policy of PAEA to afford equal opportunity in employment and contracting without regard to race, color, national origin, sex, religious preference, age, sexual orientation, genetic information, gender identity, status as a parent, disability, veteran or any other status protected by applicable US law.