Senior AI Systems Engineer: Knowledge-Grounded & Evaluation

Pearson; Washington (District of Columbia); On-site; USD 140,000 - 160,000; Full time

Job summary

Pearson seeks an experienced Lead Specialist, Applied AI Systems Engineer to design and build intelligent systems for next-gen K–12 assessment tooling. You will work across data inputs, retrieval, models, and deterministic services with strong collaboration between AI science, data architecture, and content teams.

You will develop agentic workflows, evaluate systems against ground truth, and ensure alignment with psychometrics and content standards.

Qualifications

• Advanced degree or equivalent experience in CS/AI/ML/Statistics.
• Experience designing and building end-to-end AI/machine-learning systems.
• Strong breadth across predictive ML, statistical learning, agentic and retrieval-grounded systems.
• Demonstrated experience evaluating probabilistic systems against ground truth or expert judgment.
• Knowledge graphs and knowledge-grounded AI experience to contribute to architecture.
• Strong Python proficiency and experience with TypeScript or ability to work in a TypeScript codebase.
• Track record of innovating beyond established practice with sound technical judgment.
• Ability to collaborate with engineers and domain experts to translate expertise into requirements and evaluation criteria.

Responsibilities

• Design complete, modular AI systems spanning data inputs, retrieval, models, agents, tools, deterministic services, human review, evaluation, and integration.
• Break complex problems into the right mix of ML, statistical methods, retrieval, graph reasoning, and deterministic software.
• Build working systems and reference implementations with software engineers to integrate into the assessment authoring system.
• Design architectures that evolve as data, domain knowledge, and evaluation evidence change.
• Design and implement agentic and knowledge-grounded AI workflows with models, tools, retrieval, and human decision points.
• Develop AI capabilities for content creation, verification, alignment, and analytic insights.
• Embed evaluation as a core part of development with benches, datasets, and holdout data.
• Collaborate with psychometricians, content experts, and AI scientists to translate requirements into measurable criteria.

Skills

Python

TypeScript

Collaboration

Autonomy

Education

Advanced degree in Computer Science/AI/ML/Statistics or related field

Tools

MLflow

Job description

Pearson seeks an experienced Lead Specialist, Applied AI Systems Engineer to design and build intelligent systems for next-gen K–12 assessment tooling. You will work across data inputs, retrieval, models, and deterministic services with strong collaboration between AI science, data architecture, and content teams.

You will develop agentic workflows, evaluate systems against ground truth, and ensure alignment with psychometrics and content standards.