Sr. Psychometrician, Data Center Certifications (AWS)

Amazon; Seattle (WA); On-site; USD 148,700 - 201,200; Full time

Benefits offered by this job

RSUs

Comprehensive benefits

Job summary

Amazon seeks an experienced Psychometrician to lead the measurement science behind its certification programs at global scale. You’ll ensure assessments are valid, reliable, fair, and defensible, collaborating with SMEs, instructional designers, and program managers to translate complex competencies into rigorous, scalable assessments.

In this role you will audit assessments, establish quality benchmarks, and drive standard‑setting studies, with responsibility for JTA design, item development,

Qualifications

• Master’s degree in Psychometrics, Educational Measurement, Industrial‑Organizational Psychology, or a related field.
• 5+ years of experience in psychometrics, assessment development, or credentialing programs.
• Demonstrated experience with JTA facilitation, item development, and standard‑setting methodologies.
• Knowledge of data analysis languages (R, Stata, Python)

Responsibilities

• Lead the psychometric design and validation of certification assessments across programs, including written, practical, and performance‑based formats.
• Design and audit scalable Job Task Analyses (JTAs) with SMEs and operations stakeholders.
• Apply CTT and IRT to evaluate item/test quality; conduct standard‑setting studies to establish cut scores.
• Develop and govern item development, review, and validation workflows.
• Analyze assessment data to identify bias and DIF; implement remediation strategies.
• Maintain independence between training and certification development for integrity.
• Build psychometric reporting frameworks surfacing insights on pass rates and item performance.
• Partner with third‑party assessment vendors and ensure adherence to psychometric standards.
• Communicate complex concepts clearly to non‑technical stakeholders, including senior leadership.

Skills

R

Stata

Python

JTA facilitation

Standard-setting

Assessment development

Credentialing programs

Education

Master's degree in Psychometrics, Educational Measurement, Industrial‑Organizational Psychology, or a related field

Ph.D. in econometrics, statistics, industrial engineering, operations research, optimization, data mining, analytics, or an equivalent quantitative field

Job description

Sr. Psychometrician, Data Center Certifications

Job ID: 10374451 | Amazon Data Services, Inc.

We’re looking for an experienced Psychometrician to serve as the measurement science backbone of our certification programs. In this role, you’ll ensure that every assessment we build is valid, reliable, fair, and defensible—at global scale. You’ll work at the intersection of workforce capability, assessment design, and data‑driven quality assurance, partnering closely with subject matter experts, instructional designers, and program managers to translate complex technical competencies into rigorous, scalable assessments.

In your first 90 days, you’ll complete a psychometric audit of existing assessments, establish quality benchmarks, and begin embedding rigorous measurement practices into our item development and review workflows. Within six months, you’ll be the go‑to expert on assessment validity and reliability across the program—and your fingerprints will be on every certification we launch.

Key Responsibilities

• Lead the psychometric design and validation of certification assessments across role‑based and specialty‑based programs, including written, practical, and performance‑based formats.
• Design and audit scalable Job Task Analyses (JTAs) in partnership with subject matter experts and operations stakeholders to establish defensible content blueprints.
• Apply classical test theory (CTT) and item response theory (IRT) to evaluate item and test quality; conduct standard‑setting studies (e.g., Angoff, Bookmark) to establish and validate cut scores.
• Develop and govern item development, review, and validation workflows across global content contributors, ensuring consistency and rigor.
• Analyze assessment data to identify bias, differential item functioning (DIF), and reliability issues; recommend and implement remediation strategies.
• Maintain independence between training and certification development to protect assessment integrity.
• Build and maintain psychometric reporting frameworks that surface actionable insights on pass rates, item performance, and certification quality.
• Partner with third‑party assessment vendors on test development, delivery, and quality assurance; hold vendors accountable to psychometric standards.
• Communicate complex psychometric concepts clearly to non‑technical stakeholders, including senior leadership.

A Day in the Life

Your morning starts with a fresh cup of coffee and a dataset that’s already asking questions. You review item‑level statistics from a recent pilot, flag a handful of underperforming items, then lead a standard‑setting session with SMEs, guiding them through an Angoff exercise. After lunch you present DIF analysis results to the program team, translate p‑values into plain language, and finish the day refining the content blueprint for an upcoming JTA. It’s measurement science at its most applied.

About the Team

The DC Certifications team is redefining how work readiness is measured across AWS data centers. We assess competencies tied to specific data‑center tasks, using practical and simulated environments to maintain high, consistent standards. We automate assessment delivery to keep the process simple, while ensuring independence and rigor through qualified assessors. Behind the scenes, we provide the tooling and data solutions that help sites plan and manage their workforce’s capabilities.

Basic Qualifications

• Knowledge of data analysis languages (e.g., R, Stata, Python) or other statistical software.
• Master’s degree in Psychometrics, Educational Measurement, Industrial‑Organizational Psychology, or a related field.
• 5+ years of experience in psychometrics, assessment development, or credentialing programs.
• Demonstrated experience with JTA facilitation, item development, and standard‑setting methodologies.
• Experience working with large‑scale, high‑stakes certification or licensure programs.

Preferred Qualifications

• Ph.D. in econometrics, statistics, industrial engineering, operations research, optimization, data mining, analytics, or an equivalent quantitative field.
• Experience in technical or workforce credentialing programs (e.g., trades, engineering, operations).
• Familiarity with performance‑based and simulation‑based assessment design.
• Experience managing vendor relationships in an assessment or credentialing context.
• Ability to operate effectively in a fast‑paced, globally distributed environment.

Amazon is an equal‑opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status. Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information.

The base salary range for this position is $148,700.00 – $201,200.00 USD annually. Your Amazon package will include sign‑on payments and restricted stock units (RSUs). Final compensation will be determined based on experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription), Basic Life & AD&D insurance, optional Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage, 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.