Proficiency Assessment Architect

Speak

United States

Vor Ort

USD 120.000 - 180.000

Vollzeit

14 Tage+

Eine vollständige Bewerbung in einer Minute

Maßgeschneiderter Lebenslauf und Anschreiben – versandbereit.

Schaffe es an den ATS-Filtern vorbei

Zusammenfassung

Speak is building a cutting-edge, AI-powered language learning platform and seeks an Assessment Design Lead to shape how we measure speaking proficiency. You will own constructs, rubrics, and validity plans, ensuring fairness and reliability of scores across diverse learner populations while collaborating closely with product and ML teams.

You will translate abstract goals into concrete scoring pipelines, define mastery thresholds, and help integrate AI tools without compromising quality.

Qualifikationen

  • 4+ years designing rubrics, blueprints, and item specs for a real, shipped language assessment product.
  • Understanding reliability and validity in plain language and how to catch a test measuring the wrong construct.
  • Experience ensuring fairness across diverse learner populations in assessment content.
  • Ability to translate qualitative goals into concrete scoring constructs for ML pipelines.
  • Comfort interpreting reliability statistics and basic IRT concepts for scoring systems.
  • Willingness to sign off on content validity and own the quality bar.
  • Experience using AI thoughtfully in workflow to support assessment design.
  • Ability to operate in a 0-to-1 environment and drive momentum.

Aufgaben

  • Own Assessment Design across three test types, focusing on Proficiency Test initially, with expansion to others over time.
  • Define constructs, rubrics, and blueprints for reliable scoring aligned to CEFR or equivalent standards.
  • Sign off on validity, establish mastery thresholds, and guide rubric guidelines for human labels and ML models.
  • Design validity evidence plans including benchmarking studies against external proficiency measures.
  • Collaborate with Product and ML teams on automated scoring, calibration, and feedback generation.

Kenntnisse

Psychometric design
CEFR familiarity
Fairness in testing
Qualitative to technical
Quantitative rigor
Quality ownership
AI fluency
Ambiguity tolerance

Jobbeschreibung

Speak is building a cutting-edge, AI-powered language learning platform and seeks an Assessment Design Lead to shape how we measure speaking proficiency. You will own constructs, rubrics, and validity plans, ensuring fairness and reliability of scores across diverse learner populations while collaborating closely with product and ML teams.

You will translate abstract goals into concrete scoring pipelines, define mastery thresholds, and help integrate AI tools without compromising quality.

Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

Remote Assessment Design Lead – Language Proficiency & AI
Remote Assessment Design Lead – Language Proficiency & AI

Ladders • USA

Remote
USD 108.000 - 130.000
Remote work
Competitive compensation
Impactful language assessments
Assessment Design Lead
Assessment Design Lead

Speak • USA

Vor Ort
USD 120.000 - 180.000
Assessment Design Lead
Assessment Design Lead

Ladders • USA

Remote
USD 108.000 - 130.000
Remote work
Competitive compensation
Impactful language assessments
Product Manager: AI Content at Scale
Product Manager: AI Content at Scale

Speak • San Francisco (CA)

Vor Ort
USD 150.000 - 210.000
Product Lead, Learning Tech & AI Tutor Platform
Product Lead, Learning Tech & AI Tutor Platform

Speak • San Francisco (CA)

Vor Ort
USD 210.000 - 320.000
Global offices
Office travel
Principal Product Designer - AI Language Learning
Principal Product Designer - AI Language Learning

Speak • USA

Remote
USD 140.000 - 180.000
Product Lead, AI Language Tutor Platform
Product Lead, AI Language Tutor Platform

present of AI • San Francisco (CA)

Vor Ort
USD 180.000 - 240.000
Product Manager: AI-Powered Content Generation for Learners
Product Manager: AI-Powered Content Generation for Learners

Candidate • San Francisco (CA)

Hybrid
USD 140.000 - 190.000
Language Assessment Scientist for High-Impact Tests
Language Assessment Scientist for High-Impact Tests

Duolingo • Pittsburgh

Vor Ort
USD 111.000 - 150.000
Consumer Product Designer for AI Language Learning
Consumer Product Designer for AI Language Learning

Speak • San Francisco (CA)

Vor Ort
USD 140.000 - 190.000
Global offices in SF, Ljubljana, Seoul
Travel to user regions