Language Measurement Scientist

Duolingo

Pittsburgh (Allegheny County)

On-site

USD 111,000 - 150,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Duolingo is seeking a language assessment expert to support test development and research for the Duolingo English Test. You will join our Assessment Research and Development team, collaborating across disciplines to design and evaluate innovative language assessments and ensure validity, reliability, and fairness.

You will work with measurement researchers, engineers, and designers to translate research findings into clear recommendations, publish findings, and contribute to digital assessment

Qualifications

  • PhD in language testing, applied linguistics, or related field.
  • Experience with high-stakes language assessments or similar research.
  • Expertise in language assessment, including validity, reliability, fairness, and measurement of language proficiency.

Responsibilities

  • Collaborate with measurement researchers, engineers, and designers to develop innovative, research-driven language assessments.
  • Design and evaluate assessment tasks, including new interactive item formats.
  • Conduct qualitative and quantitative studies examining validity, reliability, and fairness of assessment tasks and test scores.
  • Translate research findings into clear recommendations for technical and non-technical partners.
  • Share findings through reports, publications, blog posts, and presentations.

Skills

Project management
Organizational skills
Communication of research
Data-analysis proficiency

Education

PhD in language testing / applied linguistics / educational measurement

Tools

R
Python

Job description

Our mission at Duolingo is to develop the best education in the world and make it universally available. It’s a big mission, and that’s where you come in!

At Duolingo, you’ll join a team that cares about educating our users, experimenting with big ideas, making fact-based decisions, and finding innovative solutions to complex problems. You’ll have limitless learning opportunities and daily collaborations with world-class minds — while doing work that’s both meaningful and fun.

Join our life-changing mission to develop education for our half a billion (and growing!) learners around the world.

Read our blog to learn more.

About the role...

Duolingo is looking for a language assessment expert to support test development and research for the Duolingo English Test. You will join our Assessment Research and Development team, collaborating across disciplines to design and evaluate innovative language assessments. Your work will help ensure that assessment tasks and scores are valid, reliable, and fair while advancing Duolingo’s mission through science.

You will...
  • Collaborate with measurement researchers, engineers, and designers to develop innovative, research-driven language assessments.
  • Design and evaluate assessment tasks, including new interactive item formats.
  • Conduct qualitative and quantitative studies examining the validity, reliability, and fairness of assessment tasks and test scores.
  • Apply language assessment and measurement principles to improve test design, content, scoring, and quality assurance.
  • Translate research findings into clear recommendations for technical and non-technical partners.
  • Share findings through technical reports, research publications, blog posts, and presentations to professional and academic audiences.
You have...
  • A PhD in language testing, applied linguistics, second language acquisition, educational measurement, or a related field.
  • Experience developing or conducting research on high-stakes language assessments or comparable professional experience.
  • Expertise in language assessment, including test design, validity, reliability, fairness, and the measurement of language proficiency.
  • Experience designing and analyzing qualitative and quantitative research studies.
  • Experience communicating research through publications, reports, or presentations.
  • The ability to explain complex measurement concepts clearly to collaborators from different disciplines.
  • Proficiency with data-analysis tools such as R or Python.
  • Strong collaboration, project-management, and organizational skills.
Exceptional candidates will have...
  • Experience applying large language models, natural language processing, or related technologies to language assessment.
  • Experience collaborating with software engineers, or AI researchers to develop digital assessment products.
  • Experience designing or developing computer-adaptive tests.
  • Expertise in evidence-centered design, interactive-task development, or integrated-skills assessment.

Salary Range:
$110,500—$149,500 USD

Accommodations: We will do everything we can within reason to make sure that your interview takes place in an environment that fairly and accurately assesses your skills. If you need assistance or accommodation, please contact accommodations@duolingo.com.

Equal Employment Opportunity: Duolingo is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.

By applying for this position your data will be processed as per the Duolingo Applicant Privacy Notice.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Language Measurement Scientist
Language Measurement Scientist

Life in Pittsburgh • Pittsburgh

On-site
USD 111,000 - 150,000
Language Assessment Scientist for High-Impact Tests
Language Assessment Scientist for High-Impact Tests

Duolingo • Pittsburgh

On-site
USD 111,000 - 150,000
Proctor Quality and Learning Manager, Scaling Operations
Proctor Quality and Learning Manager, Scaling Operations

Duolingo • Pennsylvania

On-site
USD 143,000 - 193,000
Language Assessment Scientist — Shape Global Tests
Language Assessment Scientist — Shape Global Tests

Life in Pittsburgh • Pittsburgh

On-site
USD 111,000 - 150,000
Proctor Quality and Learning Manager, Scaling Operations
Proctor Quality and Learning Manager, Scaling Operations

Duolingo • Pittsburgh

On-site
USD 140,000 - 195,000
QA Specialist I, Recent Graduate
QA Specialist I, Recent Graduate

Life in Pittsburgh • New York (NY)

On-site
USD 68,000 - 78,000
Equity compensation
Senior Engineering Manager, Product
Senior Engineering Manager, Product

Duolingo • Pennsylvania

On-site
USD 204,000 - 306,000
Senior Product Manager, DET
Senior Product Manager, DET

Duolingo • Pennsylvania

On-site
USD 183,000 - 247,000
Senior Engineering Manager, Product
Senior Engineering Manager, Product

Duolingo • Pittsburgh

On-site
USD 204,000 - 306,000
Senior Engineering Manager, Product
Senior Engineering Manager, Product

Duolingo • New York (NY)

On-site
USD 204,000 - 306,000