Psychometrics Lead

Touchstone Institute; Toronto; On-site; CAD 110,000 - 150,000; Full time

Job summary

Touchstone Institute seeks a Psychometrics Lead to drive development, validation and continuous improvement of high-quality assessments. You will translate frameworks into defensible examinations, collaborate with SMEs and lead item development and scoring practices.

You will present analyses to governance bodies, ensure QA, and advance fair, reliable measurement across health professions education and regulation. This is a 24-month contract with hybrid in-office days in Toronto.

Qualifications

• Master’s degree in Statistics, Biostatistics, Quantitative Psychology or related field.
• At least 5 years of experience in an educational or testing environment.
• Experience in test development and analysis and strong psychometric knowledge.

Responsibilities

• Lead design, validation, evaluation and continuous improvement of assessments.
• Translate competency frameworks into examination blueprints and scoring approaches.
• Collaborate with SMEs to develop exam content and assessment practices.
• Prepare analyses, reports and briefing materials for governance and policy.
• Drive QA and adherence to professional standards in testing.

Skills

Time management

Analytical reporting

Verbal and written communication

Problem solving

Interpersonal collaboration

Data visualization

Leadership

Word processing and presentation

Education

Master’s degree in Statistics, Biostatistics, Quantitative Psychology or related field

Tools

Word

Excel

MS Project

PowerPoint

Job description

Position Description

Are you excited by the power of data, measurement, and continuous improvement? Do you thrive on solving complex challenges and turning insights into action? As Psychometrics Lead, you'll have the opportunity to lead innovative assessment initiatives, collaborate with passionate experts, and drive meaningful change across the health professions sector.

Bring your expertise, curiosity, and leadership to a role where your work truly matters, and your impact will be felt every day.

As Psychometrics Lead at Touchstone Institute, you will play a pivotal role in advancing fair, meaningful, and defensible assessments that support excellence in health professions education and regulation. This is an exciting opportunity to combine psychometric expertise, innovative research, and data-driven decision-making to influence how knowledge, skills, and competencies are measured across diverse assessment programs.

In this role, you will lead the design, validation, evaluation, and continuous improvement of high-quality assessment tools that accurately measure learner and test taker performance. Your work will directly contribute to the integrity, reliability, and fairness of assessments used to inform important educational and regulatory decisions.

As a trusted lead, you will collaborate closely with program teams, subject matter experts, and stakeholders across the organization. You will translate complex psychometric and assessment data into clear, actionable insights, support evidence-based quality assurance initiatives, and help strengthen assessment practices across Touchstone Institute.

Join us in shaping the future of health professions assessment and making a lasting difference in the communities we serve.

Key Accountabilities

Assessment Development & Validation

• Reporting to the Manager, Psychometrics, and working closely with the Senior Director, Assessment and Senior Director of Program Design & Implementation and relevant SMEs, develops and operationalizes clinical and written examinations and assessments that are deemed appropriate and psychometrically defensible
• Drives innovation in assessment and learning design, ensuring the development of evidence-based, valid, and impactful evaluation practices
• Translates competency frameworks and stakeholder input into comprehensive examination and assessment blueprints that support meaningful and equitable evaluation
• Designs and facilitates engaging workshops and collaborative sessions with SMEs to support examination and assessment development
• Leads and validates examination item and case development processes, ensuring alignment with psychometric principles, industry standards, and best practices
• Establishes and continuously enhances standardized approaches to candidate assessment, scoring, and performance evaluation
• Champions adherence to professional standards to ensure exams and assessments comply with industry standards (e.g. AERA, APA, NCME guidelines)
• Serves as a trusted advisorto the Assessment Oversight Committee, providing technical expertise and evidence-based recommendations.
• Prepares and presents analyses, reports, and briefing materials that support governance, policy development, and informed decision-making on assessment strategy and quality

Quality Assurance and Quality Improvement

• Leads the ongoing review of examinations, scoring systems and methodologies to ensure ongoing psychometric integrity
• Partners with cross-functional teams to transform assessment data into actionable insights that drive organizational learning, program effectiveness, and continuous improvement
• Fosters a culture to collaborate with cross-functional teams inside the organization to turn data into ideas/insights for organizational growth
• Maintains documentation/records of QA activities, audit findings, and corrective actions to support transparency and accountability

Data Analysis & Interpretation

• Conducts advanced psychometric and statistical analyses to evaluate assessment quality and generate insights that inform program design, candidate outcomes, and organizational decision-making
• Integrates research, quality assurance findings, and performance data to strengthen measurement strategies and demonstrate assessment effectiveness
• Translates complex data into clear, compelling reports, presentations, and recommendations for internal and external stakeholders.
• Develops opportunities to integrate research and quality assurance reports to inform and support strategies and demonstrate effectiveness in measurement
• Supports building networks and accessing expert advisors

Position Requirements

Knowledge

• Advanced knowledge of exam theory, learning theory, and assessment methodologies
• Strong expertise of measurement theory and methods
• Knowledge of competency-based assessment principles, particularly within health professions education and regulation
• Demonstrated expertise in examination development and test analysis
• Good judgment, critical thinking skills, and close attention to detail applied in this field

Skills

• Strong time management skills and ability to prioritize multiple assignments and work demands
• Strong analytical and report writing skills and the ability to translate statistical information for clients and regulatory bodies
• Excellent verbal and written communication skills
• Strong conceptual, analytical and problem-solving skills
• Advanced problem-solving and critical-thinking abilities, with a track record of identifying challenges and delivering practical solutions
• Strong interpersonal and collaboration skills to work with others to coordinate information
• Strong data visualization skills, and comfort applying them in new and innovative ways
• Strong leadership skills: experience managing a team of analyst staff is an asset
• Proficiency in the use of word processing, and presentation software (e.g. Word, Excel, MS Project and PowerPoint)

Abilities

• Ability to deal with emergent issues in a professional and timely manner
• Ability to work independently or as part of a team, with a range of clients at all organizational levels
• Demonstrated ability to process multiple types of information, perform multiple tasks simultaneously, and prioritize

Qualifications

• Master’s degree in Statistics, Biostatistics, Quantitative Psychology, Developmental Psychology and Education, or other equivalent degrees
• At least 5 years of experience in an educational or testing environment
• Related educational assessment and measurement experience as well as experience in test development and analysis

Job level: 6

Closing date: Monday, September 14, 2026

Notes for Candidates

• This is an existing vacancy within the organization
• This is a contractual role of 24 months
• Touchstone Institute does not use artificial intelligence (AI) tools as part of the applicant screening process. Applications will be reviewed by members of our recruitment and hiring teams to ensure a fair and thorough assessment.
• The salary range considers many factors including, but not limited to experience, education, and training, etc. It is not typical for the salary to be offered near the top of the range, it is dependent on numerous factors.
• Touchstone Institute is dedicated to employment equity and fostering a diverse work environment. We encourage applications from Indigenous Peoples, racialized persons, persons with disabilities, and other equity-seeking groups. We believe that a diverse team is essential to understanding and serving the evolving needs of the healthcare sector.

We follow a hybrid work model with two designated in-office days: Tuesdays and Wednesdays.
Successful candidates must be available to work on-site during these days at 145 Wellington St W Suite 600, Toronto, ON M5J 1H8 with or without accommodation.

Touchstone Institute is an equal-opportunity employer that is committed to fostering a diverse work environment and exercises accessible employment practices. In accordance with the Ontario Human Rights Code, Accessibility for Ontarians with Disabilities Act (AODA) and Touchstone Institute’s policies, a request for accommodation will be accepted in all aspects of our hiring process. If you anticipate needing any type of accommodation during the recruitment process, please email [e-mail] in advance of your appointment.