Psychometrician

Doist; Washington (District of Columbia); On-site; USD 120,000 - 160,000; Full time

Job summary

CAL in Washington, DC is seeking a Psychometrician to design and conduct psychometric analyses for language testing programs, including validity, reliability, equating, and scoring. Under the Director of Psychometrics and Quantitative Research, you will develop QA procedures and prepare technical reports.

The role provides quantitative guidance to researchers and coordinates external quantitative resources, supporting proposals and client collaboration with academia and education partners.

Qualifications

• Experience designing and evaluating psychometric analyses for language testing.
• Familiarity with classical test theory and Rasch-based methods.
• Ability to prepare technical reports and present results clearly.

Responsibilities

• Design and conduct psychometric analyses for performance tests including validity, reliability, and equating.
• Prepare technical reports and support quality control procedures.
• Provide quantitative guidance to researchers and coordinate external quantitative resources.
• Support outreach and collaboration with external partners and academia.

Skills

Psychometrics

Quantitative analysis

Rasch model

Winsteps

Facets software

Item response theory

Data visualization

Proposal writing support

Tools

Winsteps

Facets

Rasch modeling

Job description

Under the guidance of the Director of Psychometrics and Quantitative Research, the Psychometrician is responsible for designing and conducting psychometric analyses of test performance; conducting psychometric analyses related to test construction, validity, reliability, equating, and scoring; preparing technical reports; developing, maintaining, and applying quality control procedures; providing quantitative guidance to professional research staff at CAL; and coordinating external quantitative resources on behalf of the organization. The Psychometrician will also provide quantitative guidance to other researchers at CAL and coordinate external quantitative resources on behalf of the organization.

ESSENTIAL DUTIES:

• Design, and conduct psychometric analyses for the WIDA ACCESS programPlan, conduct, document and present psychometric and statistical analyses using classical test theory and the Rasch model with Winsteps and Facets software, including analyses related to item quality, rater performance, DIF, reliability and classification consistency, standard setting, equating, scoring, and reporting
• Investigate and evaluate new approaches to conducting psychometric analyses in language testing
• Establish and monitor quality control procedures related to quantitative analyses, from data cleaning to final technical reports, ensuring the reliability and integrity of results
• Provide guidance to professional researchers on proposal design and quantitative analytical approaches upon request and participate in writing quantitatively-oriented portions of proposals as needed
• Support outreach and collaboration, including research, development and client development activities with external partners, academia, innovators in education and language, and the public as needed