Director, AI Assisted Solutions-1

collegeboard

United States

A distancia

USD 180.000 - 250.000

Jornada completa

Hace 3 días
Sé de los primeros/as/es en solicitar esta vacante

Una candidatura completa en un minuto

Currículum y carta de presentación personalizados, listos para enviar.

Supera los filtros ATS

Ventajas ofrecidas por este puesto de trabajo

Remote work options

Descripción de la vacante

College Board is seeking a Director, Applied AI Assessment Solutions to drive AI-enabled item development across the SAT Suite and related programs. You will lead the design, tooling, prompts, and workflows, delivering scalable solutions while ensuring quality and ethical AI use in assessment creation and review.

This fully remote, full-time role emphasizes collaboration with content teams, technology, and product stakeholders to maximize ROI and minimize duplicative work across the enterprise.

Responsabilidades

  • Lead end-to-end delivery for College Board assessment program engagements: define the problem, design/build AI tools, prompts, and workflows, and deliver a working solution with a training/adoption plan.
  • Apply expert judgment to evaluate AI outputs against standards, catch misalignment, bias, and errors, and refine prompts and workflows.
  • Establish quality standards and evaluation criteria for AI-generated content, translating expertise into rubrics or scoring frameworks.
  • Identify and articulate requirements for systems-based solutions needing input from stakeholders and shape roadmaps for AI-enabled item development.
  • Diagnose gaps, document conditions for scaling enablement, and influence program roadmaps and enterprise tooling.
  • Represent program needs in cross-program discussions about shared tooling and methodology, contributing to enterprise-wide strategy.

Descripción del empleo

RAG

About the role

Assessment Design & Development (AD&D) is Learning & Assessment's content engine-the department responsible for the item and assessment development that powers the SAT Suite. AD&D is expanding its mandate: beyond sustaining operational excellence for SAT, the department is building the shared infrastructure to scale content production across programs and designing the new item types and assessment formats the future of assessments demands.Applied AI Assessment Solutions is AD&D's team dedicated to this work: leading the shared strategy, standards, systems design, and prototype tooling that let every College Board program accelerate item development without compromising quality. The team operates as an internal, enterprise-wide consultancy: Directors, Applied AI Assessment Solutions, work across College Board's assessment programs-including the SAT Suite, AP, ACCUPLACER, CLEP, and PAA-partnering closely with each program's content teams to design solutions that help programs accelerate and scale item development.As a Director, Applied AI Assessment Solutions, you translate deep item-development expertise into working AI solutions for the programs you support-applying hands-on fluency in item development and review, automated item generation (AIG), retrieval-augmented generation (RAG), prompt and context engineering, agentic workflow design, and systematic output evaluation to accelerate assessment content development across College Board's programs. Working as part of an internal consultancy, you help identify and scope the engagements where your expertise will have the greatest impact, partnering with the Executive Director and program stakeholders to assess opportunity, negotiate scope, and define what a successful engagement looks like before work begins. During engagements, you partner with a program's content team representatives to design and build point solutions that address specific production challenges. You also contribute to longer-range AI-enablement initiatives to integrate applied AI functionality into content authoring and item banking systems.Across your engagements, you'll play a crucial role in shaping how College Board's programs leverage new tools to support their content teams, ethically and effectively integrating AI technology in assessment creation and review. With your team, you work to maximize ROI and efficiency of each tool, minimizing duplicative efforts across the enterprise, while meeting each program's unique requirements.In this role, you will:Program AI Solution Design & Implementation (60%)Lead end-to-end delivery for assigned CB assessment program engagements: apply deep item development and AI expertise to define the problem and requirements, design and build the AI tools, prompts, and workflows, and deliver a working solution along with a training and adoption plan that partner teams can execute against.Apply expert judgment throughout the iterative building process-evaluating output at each iteration against the standards of an advanced domain subject matter expert, catching construct misalignment, flawed reasoning, bias, and factual errors early enough to correct course, and refining prompts, context, and workflows accordingly-and own final quality sign-off on tools before they are delivered to partner teams.Establish the quality standards and evaluation criteria used to assess AI-generated content systematically, translating domain expertise into defined rubrics, benchmarks, or scoring frameworks that others on the team can apply consistently.Identify and articulate requirements for systems-based solutions requiring input and resources from additional stakeholders.Diagnose gaps and document conditions for scaling enablement, ranging from culture and staffing, process improvements, and technological capabilities and use these findings to shape the team's engagement priorities and each program's roadmap for AI-enabled item development.Agentic Workflow Contribution (20%)Work within and across teams to define agentic workflow standards and inputs, contributing assessment expertise and on-the-ground evidence from across your program engagements to inform the broader agentic-human hybrid architecture the Applied AI Assessment Solutions team leads with partners in Technology and Digital Product.Surface patterns, gaps, and opportunities from your program engagements that inform enterprise-wide strategy and roadmaps.Represent the needs and constraints of the programs you support in cross-program discussions about shared tooling and methodology.Training, Adoption & Change Management (10%)Design and deliver traini

Remote work options

Additional Information

Director, Applied AI Assessment SolutionsCollege Board - Assessment Design and Development | Learning & AssessmentLocation: This is a fully remote role. Candidates who live near CB offices have the option of being fully remote or hybrid (Tuesday and Wednesday in office).Type: This is a full-time position

Your Match

How well this role fits your profile.

No H-1B filings for this employer appear in the public Department of Labor record. That is not the same as a refusal to sponsor - smaller employers and first-time filers often have no history, and company names do not always match the filing name. Search the sponsor database .

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Director, AI-Powered Assessment Solutions (Remote)
Director, AI-Powered Assessment Solutions (Remote)

collegeboard • EE. UU.

A distancia
USD 180.000 - 250.000
Remote work options
Senior Engineer, Agentic Engineering Governance, Open Source
Senior Engineer, Agentic Engineering Governance, Open Source

collegeboard • EE. UU.

A distancia
USD 140.000 - 170.000
Health insurance
Vision insurance
Remote work options
Lead Specialist, AI Scientist
Lead Specialist, AI Scientist

Pearson • EE. UU.

A distancia
USD 140.000 - 160.000
Director, Financial Aid Software Sales & Engagement
Director, Financial Aid Software Sales & Engagement

collegeboard • EE. UU.

A distancia
USD 150.000 - 230.000
Health insurance
Remote work options
Equity / stock options
Lead Specialist, AI Scientist
Lead Specialist, AI Scientist

Pearson • Raleigh (NC)

A distancia
USD 140.000 - 160.000
K-12 Educational Operations & Assessment Delivery
K-12 Educational Operations & Assessment Delivery

Motion Recruitment • Reston (VA)

A distancia
USD 55.000 - 83.000
Assistant Director, K-12 Delivery Specialist
Assistant Director, K-12 Delivery Specialist

Sun Technologies, Inc. • Reston (VA)

Híbrido
USD 69.000 - 76.000
Medical Insurance under ACA
401(k)
Sick leave
Assistant Director, K-12 Delivery Specialist
Assistant Director, K-12 Delivery Specialist

The Planet Group • EE. UU.

A distancia
USD 55.000 - 83.000
Education Practitioner - Learning & Assessment Specialist
Education Practitioner - Learning & Assessment Specialist

Confidential • EE. UU.

A distancia
USD 41.000 - 83.000
Senior Software Engineer (AI Applications)
Senior Software Engineer (AI Applications)

Cambium Learning Group • EE. UU.

A distancia
USD 120.000 - 160.000