Assessment Design Lead

Ladders

United States

Teletrabalho

USD 108 000 - 130 000

Tempo integral

há 21 horas
Torna-te num dos primeiros candidatos

Recebe uma resposta deste empregador

O teu currículo e carta de apresentação, adaptados exatamente ao que estão a contratar.

Ultrapassa os filtros ATS

Vantagens oferecidas por esta oferta de emprego

Remote work
Competitive compensation
Impactful language assessments

Resumo da oferta

Ladders is seeking an Assessment Design Lead to shape the strategy behind language learning assessments. The role defines what is measured, how often, and how those measures connect to learner outcomes, translating goals into clear scoreable constructs for item writers, data science, and ML partners.

You will ensure validity, fairness, and reliability across diverse learner populations while guiding responsible AI use in assessment.

Qualificações

  • 4+ years of assessment or psychometric design experience in language testing.
  • Solid knowledge of language proficiency frameworks CEFR, ACTFL, IELTS, TOEFL.
  • Ability to identify bias and ensure fair scoring across diverse populations.
  • Experience translating qualitative constructs into actionable metrics for ML.
  • Statistical analysis for reliability and validity, including inter-rater reliability.
  • Experience using AI tools in assessment design.

Responsabilidades

  • Define assessment types and their frequency, focusing on proficiency tests.
  • Translate abstract goals into scoreable constructs and rubrics for writers and ML engineers.
  • Ensure content validity across assessments and identify measurement errors before launch.
  • Design a validity evidence plan to benchmark assessments against external measures.
  • Collaborate with Product and ML teams to align constructs with automated scoring systems.

Conhecimentos

Psychometric design
Language assessments
Bias detection
AI in assessment
Reliability and validity
Translation to ML metrics

Descrição da oferta de emprego

For our client, we are seeking a Assessment Design Lead to join the team of a leader in the Education, Government & Non-Profit space.

This role will shape the assessment strategy behind language learning products, with an emphasis on defining what gets measured, how often it is measured, and how those measures connect to learner outcomes. You will translate broad educational goals into clear, scoreable constructs and rubrics that can guide item writers, data science, and machine learning partners. The position also plays a key role in ensuring assessment validity, fairness, and reliability across diverse learner populations while supporting the responsible use of AI in the assessment process.

Location: Remote

Compensation: $108,000 – $130,000 annually

Responsibilities
  • Define assessment types and their frequency, with an initial focus on proficiency tests and expansion as needed
  • Translate abstract assessment goals into concrete, scoreable constructs and rubrics for item writers and ML engineers
  • Ensure content validity across all assessments and identify potential measurement errors before launch
  • Design and implement a validity evidence plan to benchmark assessments against established external measures
  • Collaborate closely with Product and ML teams to align assessment constructs with automated scoring systems
Qualifications
  • 4+ years of experience in assessment or psychometric design for language assessments, with practical real-world application
  • Deep understanding of language proficiency frameworks such as CEFR, ACTFL, IELTS, or TOEFL
  • Ability to identify bias in assessments across diverse learner populations and ensure fairness in scoring
  • Skilled in translating qualitative constructs into actionable metrics for machine learning applications
  • Comfortable with statistical analysis related to assessment reliability and validity, including inter-rater reliability concepts
  • Proven ability to maintain high quality standards and make authoritative decisions on content validity
  • Experience using AI tools in assessment design and evaluation processes
Benefits
  • Collaborative work environment with a focus on innovation in assessment design
  • Opportunity to shape the efficacy of language learning assessments directly
  • Engagement with cutting-edge AI tools in the assessment process
  • Chance to work with a dedicated team focused on learner outcomes and assessment integrity

Our client is an equal opportunity employer. We encourage you to apply even if you don’t meet every qualification—your background could be exactly what this team needs.

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Remote Assessment Design Lead – Language Proficiency & AI
Remote Assessment Design Lead – Language Proficiency & AI

Ladders • EUA

Teletrabalho
USD 108 000 - 130 000
Remote work
Competitive compensation
Impactful language assessments
Assessment Design Lead
Assessment Design Lead

Speak • EUA

Presencial
USD 120 000 - 180 000
Proficiency Assessment Architect
Proficiency Assessment Architect

Speak • EUA

Presencial
USD 120 000 - 180 000
Sr. Product Manager, Assessments
Sr. Product Manager, Assessments

Language Testing International • EUA

Teletrabalho
USD 130 000 - 170 000
Education Practitioner - Learning & Assessment Specialist
Education Practitioner - Learning & Assessment Specialist

Confidential • EUA

Teletrabalho
USD 41 000 - 83 000
Lead Specialist, AI Scientist
Lead Specialist, AI Scientist

Pearson • Augusta (ME)

Presencial
USD 140 000 - 160 000
Lead Specialist, AI Scientist
Lead Specialist, AI Scientist

Pearson • Oklahoma City (OK)

Presencial
USD 140 000 - 160 000
Lead Specialist, AI Scientist
Lead Specialist, AI Scientist

Pearson • Springfield (IL)

Presencial
USD 140 000 - 160 000
Lead Specialist, AI Scientist
Lead Specialist, AI Scientist

Pearson • EUA

Teletrabalho
USD 140 000 - 160 000
Lead Specialist, AI Scientist
Lead Specialist, AI Scientist

Pearson • Jackson (MS)

Presencial
USD 140 000 - 160 000