Principal Psychometrician

Houghton Mifflin Harcourt Co; Boston (MA); On-site; USD 120,000 - 180,000; Full time

Job summary

Houghton Mifflin Harcourt is seeking a Principal Psychometrician to join the Learning Sciences, PRDA team, and lead psychometric research for K-12 assessment products across Core and SIS lines. Responsibilities include CAT design, IRT calibration, growth modeling, and data analytics to improve teaching and learning experiences for students and teachers.

The ideal candidate will have a Doctorate in psychometrics or related field, 5+ years in testing, and strong programming skills in R, with

Qualifications

• Doctorate in psychometrics, educational measurement, statistics, or closely related field.
• 5+ years in a testing-related field solving psychometric problems; strong measurement knowledge.
• Experience with computer adaptive testing and multi-stage testing preferred.
• Experience with R Shiny app development and RMarkdown preferred.

Responsibilities

• Lead statistical analyses for design, development, and interpretation of assessment solutions.
• Develop computer adaptive test design, blueprint evaluation, and test assembly.
• Manage large and complex data files, apply statistical software (R), and use IRT calibration tools.
• Conduct analyses and create visual displays to interpret data and lead CAT system development.
• Lead simulation studies for various measurement issues.
• Identify risks in data and analyses and develop mitigation plans.
• Present technical test characteristics to internal and external stakeholders.
• Collaborate across teams to coordinate multi-project work and manage schedules.

Skills

IRT calibration

R programming

Shiny apps

SAS

WINSTEPS

FlexMIRT

R packages

Cloud computing

Office/Excel

Bayesian stats

Large data handling

Education

Doctorate in psychometrics

5+ years experience in testing field

Tools

R

SAS

WINSTEPS

FlexMIRT

Job description

Select how often (in days) to receive an alert: Create Alert

Additional Locations: Location 1Location 2

HMH is looking for a Principal Psychometrician to join the Learning Sciences’ Psychometric Research & Data Analytics (PRDA) team .

The Principal Psychometrician will assist in leading projects involving the design, development, and implementation of various assessment products for HMH that will help optimize the teaching and learning experiences of HMH’s students and their teachers. The Principal Psychometrician will contribute to, and be accountable for and assist with, all technical aspects of assigned projects, including computer adaptive test design, item and test analyses, growth modeling, mastery model development, IRT calibration, equating, scaling, prediction studies, aggregated and summative assessment data analyses, and norms development across SIS and Core product lines. Expertise with statistical software is required (preferentially, R). Expertise with IRT-related software (e.g., Winsteps, FlexMIRT, various R packages) is also required. A moderate- to high-degree of expertise with large data is also preferred.

Overall, the Principal Psychometrician position will support HMH’s assessment design, psychometric analyses, interim and program assessment evaluation, reporting, and growth research to further the goals of HMH in creating, maintaining, and enhancing a connected teaching and learning system of services and products (in both Core and SIS lines of business).

Candidates interested in this position should have an expert knowledge of and interest in the assessment challenges facing K-12 education, a collaborative working style, expertise in psychometric modeling, statistical programming, research methodology, and a keen interest in supporting data analytic initiatives.

Primary Responsibilities

• Work with the Manager, Psychometric Research & Data Analytics, other psychometricians, subject-matter experts within the Assessment Solutions & Design team, engineering team who implement PRDA-designed CAT algorithms, and/or clients/customers to select and perform statistical analyses that will best support the design and development of meaningful assessment solutions for HMH’s students and their teachers.
• Develop computer adaptive test design, blueprint evaluation, and test assembly.
• Manage large and complex data files, apply statistical analysis software, such as R, and utilize IRT calibration software (e.g., Winsteps, FlexMIRT, and dedicated R packages) to evaluate and model these data.
• Conduct analyses and create visual displays to lead interpretation of the data. Lead the development of computer adaptive testing systems, which includes multistage adaptive designs and automated test assembly.
• Develop and implement simulation studies investigating assorted measurement issues.
• Initiate, plan, and select, appropriate statistical and psychometric methodologies involved in the accurate scoring, analysis, and reporting of assessment programs. Identifies issues and risks in data and psychometric analyses and creates plans to mitigate the risk. Leads efforts on quality assurance and quality checking procedures.
• Initiate and lead procedures for presenting technical characteristics of tests and evaluate the extent to which product development activities will result in sound assessment products. Make recommendations to improve the robustness of assessments.
• Investigate and build quality control mechanisms or processes that help ensure the consistent production of valid and reliable items, tests, and reports.
• Lead the development of project and research specifications.
• Independently implement a research study design and develop and follow analyses specifications.
• Serve as the point of contact for internal and external customers regarding the technical characteristics of current tests or those under development.
• Collaborates with colleagues to coordinate work with large or multiple projects and to negotiate schedules with Engineering or project management staff.
• Demonstrates project management skills necessary to lead all Psychometric Research & Data Analytics aspects of large operational assessment programs or research projects.
• Demonstrates communication skills (oral, written, and business) necessary to function effectively as a primary contact with internal project teams and external clients.

Ideal Candidate

The ideal candidate for this dynamic role enjoys solving challenging assessment problems through collaborative engagement across multiple assessment and analytic teams and has diverse experiences that can support both the technical phases of the asset development process ( e.g., development of CAT test design and algorithm specifications, sampling design, test form/item pool construction, item and test analyses, IRT calibration, equating and scaling activities, reliability and validity analyses, etc. ).

The ideal candidate will also possess outstanding programming skills to assist with managing data out of HMH’s educational assessment platform. It would be an additional plus if the candidate has applied experience in performing psychometric analyses for computer adaptive assessment designs (including multistage), technology enhanced items, innovative assessment types (e.g., NGSS Science), and interest in machine learning applications of assessment data.

Education and Experience

• Doctorate in psychometrics, educational measurement, statistics, or closely related field of study
• 5+ years’ experience working in a testing-related field with demonstrable experience in solving psychometric problems and providing solutions. Sound conceptual understanding of psychometrics and measurement is essential. Demonstrated professional achievement through presentations at professional conferences and/or publications in professional journals is a plus .

Knowledge and Skills

• Knowledge of and experience with IRT calibration, scaling, equating, norming, program assessments, and growth measures.
• Experience with computer adaptive testing. Experience with multi-stage testing and/or automated test assembly is preferred.
• Experience in R Shiny app development and RMarkdown is preferred.

Knowledge of technical requirements, test statistics, and APA/AERA Standards and ability to perform psychometric analyses that meet all professional standards of test development and score uses.

• Knowledge of support tools such as Microsoft Office/EXCEL, e xpertise with computer software packages commonly used in psychometric work and research (e.g., R , SAS, WINSTEPS, FlexMIRT).
• Strong R programming skills and experience with IRT-related R packages is preferred. Knowledge of Bayesian statistical methods is preferred
• Demonstrated ability to analyze large data sets efficiently and experience with cloud computing tools and analytic programming languages is a plus
• Ability to apply knowledge gained through experience in test development to prepare project specifications, anticipate implementation problems, and lead projects through to completion
• Demonstrate strong organizational and communication skills and be able to function independently and collaboratively on research projects
• Ability and willingness to adapt test theory for practical solutions
• Approaches testing and assessments problems with an analytical, flexible, and investigative mind
• Strong organizational and communication skills

Physical Requirements

• Might be in a stationary position for a considerable time (sitting and/or standing).
• The person in this position needs to move about inside the office to access file cabinets, office machinery, etc.
• Constantly operates a computer and other office productivity machinery, such as a calculator, copy machine, and computer printer.
• Must be able to collaborate with colleagues via face to face, conference calls, and online meetings.

ABOUT US:

Houghton Mifflin Harcourt (NASDAQ:HMHC) is a global learning company dedicated to changing people’s lives by fostering passionate, curious learners. As a leading provider of pre‑K–12 education content, services, and cutting‑edge technology solutions across a variety of media, HMH enables learning in a changing landscape. HMH is uniquely positioned to create engaging and effective educational content and experiences from early childhood to beyond the classroom. HMH serves more than 50 million students in over 150 countries worldwide, while its award‑winning children’s books, novels, non‑fiction, and reference titles are enjoyed by readers throughout the world.
For more information, visit http://careers.hmhco.com

PLEASE NOTE:

Houghton Mifflin Harcourt is an equal employment opportunity employer and participates in E-Verify. All qualified applicants will receive consideration for employment and will not be discriminated against on the basis of gender, race/ethnicity, gender identity, sexual orientation, protected veteran status, disability, or other protected group status.

Nearest Major Market: Boston
Job Segment: Market Research, Education, Marketing, Publishing