AI Quality & Pedagogy PM for Multilingual Learners

Workshop Hiring

Boston (MA)

Hybrid

USD 120,000 - 180,000

Full time

9 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Medley Learning is hiring a Technical Product Manager, Pedagogy & AI Quality to ensure AI-powered instructional scaffolds meet high quality and user experience standards at scale. You’ll define quality metrics, expand eval systems, and work with engineering to ship improvements that benefit multilingual learners and students with disabilities.

You'll collaborate with the head of product and engineers to own the quality and usability of our scaffolds, prioritizing features that enhance teacher

Qualifications

  • Pedagogical expertise in scaffolding literacy for multilingual learners and special education.
  • Experience with AI evals including labeling, rubric design and error analysis.
  • AI product fluency with daily use of LLMs and designing AI-driven systems.
  • Ability to define features and collaborate with engineers to deliver them.

Responsibilities

  • Define what “excellent” looks like and translate research into quality standards for Medley scaffolds; set AI output targets.
  • Track quality with a robust evals system; build datasets, rubrics, eval suites and perform error analysis.
  • Refine prompts and models; run model comparisons and regression tests to meet quality thresholds.
  • Stay close to users; review educator signals, support data, product analytics and conduct user research to inform decisions.
  • Prioritize and ship AI quality and scaffold UX improvements; write specs and collaborate with engineering on fixes and new features.

Skills

Pedagogical expertise
AI evals
AI product fluency
Feature definition & UX
Clear writing & communication

Education

Graduate degree in education or related field

Tools

LangSmith
Braintrust
Arize
Langfuse

Job description

Medley Learning is hiring a Technical Product Manager, Pedagogy & AI Quality to ensure AI-powered instructional scaffolds meet high quality and user experience standards at scale. You’ll define quality metrics, expand eval systems, and work with engineering to ship improvements that benefit multilingual learners and students with disabilities.

You'll collaborate with the head of product and engineers to own the quality and usability of our scaffolds, prioritizing features that enhance teacher

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Technical PM: Pedagogy & AI Quality
Remote Technical PM: Pedagogy & AI Quality

TrulyHired • United States

Remote
USD 130,000 - 150,000
AI Quality & Pedagogy Product Manager — Multilingual Literacy
AI Quality & Pedagogy Product Manager — Multilingual Literacy

Workshop Venture Partners • Boston (MA)

Hybrid
USD 130,000 - 150,000
AI-Pedagogy Quality Product Manager
AI-Pedagogy Quality Product Manager

Workshop Hiring • United States

Hybrid
USD 90,000 - 120,000
Technical Product Manager, Pedagogy & AI Quality
Technical Product Manager, Pedagogy & AI Quality

TrulyHired • United States

Remote
USD 130,000 - 150,000
Technical Product Manager, Pedagogy & AI Quality
Technical Product Manager, Pedagogy & AI Quality

Workshop Hiring • United States

Hybrid
USD 90,000 - 120,000
Technical Product Manager, Pedagogy & AI Quality
Technical Product Manager, Pedagogy & AI Quality

Workshop Venture Partners • Boston (MA)

Hybrid
USD 130,000 - 150,000
Product Manager: AI-Powered Content Generation for Learners
Product Manager: AI-Powered Content Generation for Learners

Candidate • San Francisco (CA)

Hybrid
USD 140,000 - 190,000
Technical Product Manager, Pedagogy & AI Quality
Technical Product Manager, Pedagogy & AI Quality

Workshop Hiring • Boston (MA)

Hybrid
USD 120,000 - 180,000
Product Manager - AI Content Gen & Quality Systems
Product Manager - AI Content Gen & Quality Systems

We Love AI Jobs • San Francisco (CA)

Hybrid
USD 191,000 - 315,000
Product Manager: AI Content at Scale
Product Manager: AI Content at Scale

Speak • San Francisco (CA)

On-site
USD 150,000 - 210,000