Lead AI Systems Engineer for Educational Assessment

Pearson; Carson City (NV); On-site; USD 140,000 - 160,000; Full time

Job summary

Pearson seeks a Lead Specialist, Applied AI Systems Engineer to design, build, and continuously improve intelligent systems for next‑gen K–12 assessment and learning.

Based in Pearson’s Measurement & Learning team, you’ll work across Content Development, Learning Science, Technology, AI Science, and Psychometrics to translate domain expertise into testable AI capabilities within our assessment authoring system.

Qualifications

• Advanced degree in Computer Science, Artificial Intelligence, Machine Learning, Statistics, or a related field, or equivalent experience
• Experience designing and building complete AI or machine-learning systems end to end, not only isolated models or prompts
• Strong breadth across predictive ML, statistical learning, and LLM-based, agentic, and retrieval-grounded systems
• Demonstrated experience evaluating probabilistic systems against ground truth or expert judgment, including evaluation datasets, error analysis, and regression testing
• Experience with knowledge graphs and knowledge-grounded AI sufficient to contribute to architecture decisions
• Strong Python proficiency, and experience with TypeScript or the ability to become productive quickly in a TypeScript codebase
• Track record of innovating beyond established practice, with sound technical judgment and high autonomy
• Ability to collaborate closely with engineers and domain experts, translating expertise into requirements and evaluation criteria

Responsibilities

• Design complete, modular AI systems spanning knowledge and data inputs, retrieval, models, agents, tools, deterministic services, human review, evaluation, and integration
• Break complex problems into the right combination of predictive ML, statistical methods, generative models, retrieval, graph reasoning, deterministic software, and human decision points
• Build working systems and reference implementations, and partner with software engineers to bring them into the assessment authoring system
• Design architectures that can evolve as models, data, domain knowledge, and evaluation evidence change
• Design and implement agentic and multi-step workflows using models, tools, retrieval, state, structured outputs, and human decision points
• Build knowledge-grounded systems that use graph, semantic, vector, and hybrid retrieval as appropriate
• Co-design knowledge and graph architecture with the AI Data & Knowledge Architect, contributing substantively rather than treating the knowledge layer as a black box
• Develop AI capabilities for content creation and revision, verification, alignment across frameworks, guidance, and analytic insights
• Make evaluation part of how systems are built, not a final step
• Build and maintain evaluation benches, including expert-adjudicated datasets, challenge sets, regression suites, and protected holdout data
• Partner with psychometricians and content experts to turn domain expectations into measurable criteria
• Analyze failure modes, critical errors, and performance across meaningful data and task slices, not only aggregate metrics
• Monitor deployed systems for drift and emerging failures, and use findings to drive improvement
• Develop predictive models, classifiers, and ranking methods where they outperform generative AI
• Evaluate tradeoffs among prompting, retrieval, structured knowledge, model selection, and model adaptation, without locking the architecture to a single provider
• Apply parameter‑efficient fine‑tuning and other adaptation methods when evaluation evidence shows they add value
• Turn expert judgments and adjudicated examples into versioned evaluation and training assets
• Design synthetic-data approaches with rigorous quality controls, never treating generated data as ground truth by default

Skills

AI systems design

Python

TypeScript

Evaluation against ground truth

Education

Advanced degree in Computer Science / AI / ML / Statistics

Tools

Go

Python (programming)

TypeScript

Job description

Pearson seeks a Lead Specialist, Applied AI Systems Engineer to design, build, and continuously improve intelligent systems for next‑gen K–12 assessment and learning.

Based in Pearson’s Measurement & Learning team, you’ll work across Content Development, Learning Science, Technology, AI Science, and Psychometrics to translate domain expertise into testable AI capabilities within our assessment authoring system.