External Evaluation

Lilly and Company; Deutschland; Remote; EUR 40.000 - 60.000; Vollzeit

Zusammenfassung

A leading evaluation consultancy is seeking an evaluator to assess the effectiveness of science engagement initiatives among faith leaders in Germany. The role involves developing evaluation methodologies, collecting data, and providing actionable recommendations. Candidates should have experience in evaluation research and familiarity with the intersections of science and faith. This position offers a competitive budget for project expenses, adhering to stringent deadlines.

Aufgaben

• Conduct a real time evaluation of the SECF Initiative's impact.
• Produce recommendations for program improvement.
• Develop new survey tools and revise historical surveys.

Jobbeschreibung

DEADLINE FOR SUBMISSIONS: 8 August 2025

Click Here for the complete RFP pdf with all details, deadlines, and supplemental material.

Organization: Science Engagement for Theological Education Initiatives of the American Association for the Advancement of Science (AAAS) Dialogue on Science, Ethics, and Religion (DoSER) program.

Project: Conduct a real time evaluation of the impact of the Science Engagement for Congregational Flourishing (SECF) Initiative and related efforts in science engagement for faith leaders. In addition to the evaluation of the SECF Initiative, the chosen evaluator will produce recommendations for program improvement going forward, including a review of the program theory of change and logic model, and an ongoing monitoring and evaluation plan.

Timeframe: September 2025 – July 2026.

Background: From 2017 to 2018, DoSER carried out a pilot project in “Science in Continuing Education for Pastors” aimed at integrating scientific and technological topics relevant to pastoral ministry into continuing education courses. This project worked with four seminaries that had been involved in DoSER’s successful “Science for Seminaries” project to develop new courses that could reach pastors with little to no previous science exposure and provide them resources and guidance in discussing scientific topics and issues with their congregants. Approximately 565 clergy and church leaders participated in courses and events developed by participating seminaries including continuing education courses such as “Theology and Medicine,” “Your Spiritual Brain,” and “Big Bang and Cosmic Creation,” along with workshops and events on “Holy Curiosity! Science, Wonder, and the Desire to Know,” “What Pastors Need to Know About Theology in a Scientific World,” and “Clergy Burnout and Coping.” Pre- and post-program surveys and interviews found that by the end of their respective programs, participants reported an increased interest in science, a greater likelihood of including science in their congregational engagement, and a desire to further explore and directly address the intersection of science and faith in future learning experiences, among other positive outcomes.

The SECF initiative will build on this successful pilot and expand both its scope and reach to improve continuing education opportunities for congregational leaders and nurture practices and resources that enhance congregational flourishing through theological engagement with science. The project involves two overlapping phases, involving working both with providers of continuing education programs for congregational leaders and directly with congregational leaders themselves, including both clergy and laity. The work with continuing education providers will help inform the programming for congregational leaders and vice versa.

Additionally, the Climate Science in Theological Education (CSTE) initiative provides grants for theological education institutions to incorporate engagement with climate science into the training of future faith leaders. Grant recipients carry out a variety of activities, including course revisions, public events, faculty workshops, etc.
The evaluation study should be designed with two stakeholder groups in mind. The first are the funders of the SECF & CSTE Initiatives – Lilly Endowment & the David and Carol Myers Foundation. For this group,it will be important to demonstrate the impact grant-funded projects have had on project leaders and project participants (congregational leaders, lay leaders, and congregants). The second stakeholder group is the staff at AAAS, as the evaluation will provide guidance for program improvements going forward and enable the staff to maximize Initiative’s impact on continuing education with science engagement and preparation of project participants willingness and comfortability to engage with science in their own context. It could help identify promising areas of future growth for the program. It is important that the methodology and written deliverables be credible to our scientific society and funder as well as accessible to a more generally educated audience.

Details about the AAAS mission and history are available at https://www.aaas.org/mission.
Details about DoSER are available at https://www.aaas.org/doser.

Project Outcomes: The focus of this study will be to ensure that we are making meaningful progress towards the outlined outcomes of the initiatives, to adjust and enhance project activities to respond to the experience of participants, and to make sure we are reaching our medium-term and long-term outcomes of the logic model, identified as:

1. Continuing education grant application process:

a. At least 25 continuing education providers apply for grants.
b. Applications and awards represent a theologically, geographically, racially, and culturally diverse population, and we expect at least 50 applications.

2.Continuing education grant activities

a. Grantees will indicate increased scientific knowledge and comfort, engagement with the network of other grantees, and development of new ideas for science-engagement in continuing education and ministry and we expect between 8-16 grantees.
b.Programs funded by continuing education grants address a variety of scientific topics and utilize a variety of approaches to engaging science in ministry contexts. These programs will be conducted through various models and will offer these programs to potentially 100-1000 participants.
c.Grantees will participate actively in cohort activities and the online networking community. This will include grantees, science advisors, and SECF advisory committee members.
d.Participants in grant-funded programs represent a diverse population of congregational leaders, approximately 100-500 participants.
e.Participants in grant-funded programs initiate science-engaged ministry projects within their own congregations. There may be up to 500 participants in this part of the grantee’s projects.
f.Participants will increase in interest in science topics, in familiarity with connections between science topics and faith and theological understanding, and in comfort and willingness incorporating science engagement into their ministry. This group may include up to 500 participants who may be surveyed.
g.AAAS members and other scientists will readily participate in project activities and will continue to engage with these programs beyond the formal grant period. There will be at least 16 scientists participating in an advisory role and at least an additional 40 scientists participating in other grantee’s program activities.

3. Congregational leadership workshops

a. Participation in congregational leadership workshops will include representatives of theologically, racially, economically, and culturally diverse communities. There will be at least 50 congregation leaders in virtual workshops and at least 60 congregational leaders in in-person workshops.
b.Workshop participants will indicate increased scientific knowledge and comfort, increased commitment to incorporating science engagement into congregational ministry, and increased confidence in strategies and approaches for science-engaged ministry.
c.Workshop participants will join and participate in the online networking community. This will include 45-110 congregational leaders.
d.Workshop participants will initiate science-engaged programs in their congregations.

4. Congregational mini-grants

a. At least 75 congregational leaders will apply for mini-grants for 45 mini-grant opportunities.
b.Mini-grant recipients will participate actively in cohort activities and the online networking community. At least 45 congregational leaders will participate in networking communities.
c.Congregations that participate in science-engagement activities will continue and expand these efforts.
d.Congregations that participate will indicate increased congregational engagement, new participants and visitors taking part in activities, and greater appreciation and understanding of the connections between science and faith.

5. Resource development

a. Continuing education providers will create new resources for inspiring science engagement in congregational leadership, including courses, events, online education tools, etc. There may be at least 24 resources developed by continuing education providers.
b.DoSER will create resources for public use by congregational leaders including best practices guides, toolkits, podcasts, videos, etc. DoSER will develop at least 12 resources from grantee’s activities and consultation.
c.Downloads of and engagement with resources on DoSER’s websites will increase year-to-year over the course of the project and beyond.

6. Support network

a. DoSER will establish accessible tools for online networking and resource sharing for project participants.
b.Project participants, including both continuing education providers and congregational leaders will participate actively in the online networking community.
c.Project participants will indicate the value of networking tools for sharing resources, asking questions, and providing support. At least 61 project participants will be included in community networks.
d.Participants in the culminating retreat will indicate continued interest in science engagement, ideas for future initiatives, and connections to other potential participants. There will be at least 45 participants in the culminating retreat.

7. Climate Science in Theological Education Grant Activities

a. Participants in grant-funded programs will reflect an increase in interest in climate science topics, in familiarity with connections between climate science topics and faith and theological understanding, and in comfort and willingness incorporating climate science engagement into their ministry.
b.Participants in grant-funded programs will initiate climate science-engaged ministry projects within their own congregations.

Program management going forward: In addition to the above initiative outcomes, we wish to explore the following questions with regard to future efforts:

1. How might DoSER be improved to better accomplish its mission?
2. How can the evaluation data be used to inform future project proposals, external and internal collaboration efforts, and securing program funding?

Primary and Secondary focus of Proposed Evaluation
The primary focus of the evaluation will be to measure the impact of initiative activities on program participants and their congregations. In addition, the evaluation will provide guidance for program improvements going forward and enable DoSER staff to maximize the Initiatives’ impact on theological education with science engagement and preparation of project participants willingness and comfortability to engage with science in their own context. It could help identify promising areas of future growth for the program.

It is important that the methodology and written deliverables be credible to our scientific society and funder as well as accessible to a more generally educated audience. DoSER staff will partner with contracted evaluator to develop new survey tools, revise historical survey tools, and when necessary conduct virtual interviews. The evaluation need not be limited to these proposed evaluation questions listed below.

Primary focus Evaluation Questions:
1. To what extent are continuing education providers, congregational leaders, workshop participants, and other pertinent project participants comfortable with engaging with science and technology in their contexts?
2. To what extent are continuing education providers, congregational leaders, workshop participants, and other pertinent project participants willing to engage with science and technology in their contexts?
3. To what extent do continuing education providers, congregational leaders, workshop participants, and other pertinent project participants change their knowledge of science and technology topics and their attitudes on the importance of engaging with these topics in their contexts as a result of program participation?
4. How do CSTE and SECF project leaders view the impact, effectiveness, and sustainability of funded projects?
5. What is the impact on congregations of engagement activities initiated by project participants?
6. What is the impact of workshops (virtual and in-person) on participants level comfortability with engaging with science and technology topics?
7. What is the relevance and impact of DoSER and grantee developed resources?
8. To what extent are the network tools and resources accessible and useful to project participants?

Secondary Focus Evaluation Questions:
1. What are the impacts of participation in initiative activities on scientists and science advisors who work with participants?
2. How can the application process for all grantees be improved?
3. What are project participants’ topical interests, source of science and technology news/resources, and demographic information?
4. How can the impact and sustainability of initiative activities be broadened beyond direct participants?

Data Collection: Data can be collected via a variety of means from SECF & CSTE grantees and project participants, scientists, and other designated stakeholders. DoSER will supply initial lists of names of individuals in various groups, as well as current known contact information. Data collection could include:
Review of historical data from “Continuing Education for Pastors Pilot Project”.

Review of previous pre- and post-course surveys and development or revision for current projects

Review of previous post-workshop surveys and development or revision for current projects.
Interviews with selected grantees, project participants, scientists, clergy, lay leaders, and congregants.
Focus groups with selected grantees, project participants, scientists, clergy, lay leaders, and congregants.
Review of other project evaluation data (Science for Seminaries, Continuing Education for Pastors Pilot Project, etc.).
Initiative network analysis.
Offerees are encouraged to identify data collection methods in their proposals. No particular method is prescribed by DoSER. Upon receiving the award, the offeree will have access to existing SECF & CSTE data, as well as historical Continuing Education for Pastors Pilot Project for analysis as needed.

Deliverables:
For AAAS Staff:

1. Overall project plan, with milestones and key dates.
2.A plan for the evaluation that addresses the logic model Outcomes and Effects.
3.Draft survey and focus group questions for DoSER review before survey or focus group implementation.
4.A strategic learning report, which would outline recommendations collectively to the whole program, and recommendations for project development. It would also pose additional questions for future analysis.
5.A review of the DoSER theory of change and an updated logic model, as appropriate.
6.A plan for future monitoring and evaluation which addresses the collection of base level data annually (assessing current DoSER reporting/assessment surveys) and obtaining data in samples from alumni at specific intervals to determine both short- and long-term impact of the Initiative. Appropriate intervals would be determined by DoSER.
7.Monthly status reports tracking progress and budget.
8.Copies of all data collected or generated during the evaluation or related efforts.
9.Guidance on publishing evaluation findings in a peer-reviewed periodicalContacts/Resources: Consultant(s) will report to Curtis Baxter, Project Director of the SECF and CSTE Initiatives, and will have as an additional point of contact Leif Castren, DoSER Program Associate. Project could involve interaction with members of the DoSER staff; staff in other parts of AAAS; designated current and alumni project participants; selected funder liaisons and Points of Contact; and scientists.

Proposal Submission: Submission package should be no more than 10 pages, to include the following items in the following order. Proposals that do not adhere to these specifications may be penalized and/or disqualified.

1.A completed title page including the following information: full name, title, and company (if applicable), mailing address, phone number, and email address. Please include the links to any websites and/or social media accounts that may be referenced for additional information.
2.Description of the proposing firm, including background and high-level corporate expertise relevant to this project (1 page limit).
3.Proposed plan and methodology for this study, including a justification of the choice of methodology based on scholarly literature and current best practices. Include a description of anticipated final products.
4.Budget. The budget for the study must be submitted along with a budget narrative that explains costs per each line item, and includes key personnel, labor hour estimates, labor rates, estimated travel expenses, and any materials cost. Budget should include breakdowns and
hours for specific processes and deliverables. Travel expenses are required to follow U.S. federal government travel regulations. Budget should not exceed $30,000 USD.
5.List of three professional references. You may also include relevant previous clients in this section.
6.Appendix A - Up to three samples of applicant’s prior evaluation work (not included in the 10-page limit).
7.Appendix B – Resumes of Key Personnel to be involved in this project (not included in the 10-page limit).

The chosen evaluator will hold quarterly meetings with DoSER, additionally, there will be a meeting within the first sixty days of the project to formulate questions and arrive a deeper understanding of the program, and one no later than 30 days prior to the end of the contract to share findings and recommendations. If necessary, travel costs for in-person meetings should be built into the budget, but virtual meetings are also welcome.

Upon award, the offeror will be subject to applicable data privacy safeguarding requirements to protect any accessed personally identifiable information (PII). Awardee will be subject to Federal Acquisition Regulations as applicable.

Proposals will be evaluated on the basis of:
Adherence to guidelines set out in this document.
For any questions or additional information about the SECF or CSTE projects contact Curtis L Baxter