Evaluation Associate

City of Philadelphia; Philadelphia (Philadelphia County); On-site; USD 63,000 - 75,000; Full time

Job summary

City of Philadelphia seeks a Data and Evaluation Associate to support large-scale program evaluations and performance reporting for Adult Education and MOE initiatives. You will design data tools, manage data sources, and produce timely reports for program leaders and stakeholders.

The role involves collaborating with PMT and MOE staff on-site, ensuring data quality, and advancing network-wide evaluation efforts through CQI processes.

Qualifications

• Bachelor's degree required.
• 2+ years data analytics and evaluation design experience preferred.
• Strong knowledge of quantitative and qualitative data analysis.

Responsibilities

• Prepare and submit quarterly MOE data reports for internal and public audiences.
• Generate graphs, tables, and other data visuals.
• Assist in survey design, administration, and analysis.
• Automate regular data reporting of administrative, compliance, and quality data.
• Monitor contract compliance by coding and maintaining provider-specific monthly reports.
• Develop written reports and presentations for a variety of audiences.
• Support Continuous Quality Improvement processes across the AEPHL network.

Skills

R

STATA

SPSS

Qualtrics

Python

Data visualization

Excel

Written communication

Analytical skills

Education

Bachelor's Degree in Social Policy / Government Administration or related

Tools

Python

R

STATA

Qualtrics

Job description

• Minimum Education Required: High School/GED
• Compensation: USD 63,000 - USD 75,000 - yearly

Company Description

A best-in‑class city that attracts best-in‑class talent, Philadelphia is an incredible place to build a career. From our thriving arts scene and rich history to our culture of passion and grit, there are countless reasons to love living and working here. With a workforce of over 30,000 people, and more than 1,000 different job categories, the City of Philadelphia offers boundless opportunities to make an impact.

As an employer, the City of Philadelphia values inclusion, integrity, innovation, empowerment, and hard work above all else. We offer a vibrant work environment, comprehensive health care and benefits, and the experience you need to grow and excel. If you’re interested in working with a passionate team of people who care about the future of Philadelphia, start here.

What we offer

• Impact- The work you do here matters to millions.
• Growth- Philadelphia is growing, why not grow with it?
• Diversity & Inclusion- Find a career in a place where everyone belongs.
• Benefits- We care about your well-being.

Agency Description

This position is joint posting with the Philadelphia Office of Children and Families (OCF) Division of Performance Management and Technology (PMT) and the Mayor's Office of Education (MOE). OCF supports Philadelphia's most vulnerable children and families and prioritizes safe children, strong families and supported schools. OCF administers a number of City-supported prevention programs such as Out-of-School Time and Career Connected Learning.

The Mayors Office of Education brings a unified vision and organizing strategy to advance Mayor Parked s vision of a world-class system of education for Philadelphians of all ages and socioeconomic backgrounds. MOE develops policies and programs, implements mayoral initiatives, and facilitates collaboration across city departments and education institutions to support success for Philadelphia learners from kindergarten through adulthood. MOE administers programs that include Adult Education, Community Schools, Extended Day, Extended Year and Foster Grandparents and advances initiatives to support teacher recruitment and retention, and college and career readiness.

The Division of Performance Management and Technology supports the core missions of OCF and MOE by:

• Building a world class data and information technology infrastructure;
• Developing and maintaining integrated information systems and data analytics so that practitioners can easily access and utilize quality and accurate data in their day-to-day decisions;
• Evaluating service quality and consistency, identifying needed improvements, and tracking these improvements over time.

Job Description

Position Summary

The Data and Evaluation Associate's primary responsibilities are to: 1) prepare regular reports on key programs to for program leaders and administrators; 2) support the design of rigorous mixed methods program and agency evaluations (integrating qualitative and quantitative data); 3) help design and administer program evaluation tools that measure program compliance and quality; 4) monitor programs' quality and compliance; 5) analyze administrative, compliance, and quality data; 6) design, test, and administer stakeholder surveys. The Associate will divide their time equally between PMT. where they will be supervised by a Data and Evaluation Manager, and the Mayor's Office of Education (MOE), where they will work on-site with program staff.

Essential Functions

1) Support the design and implementation of research and evaluation projects that advance program and system performance improvements for Adult Education, Extended Day Extended Year and other MOE programs.

2) Clean, manage, and integrate data from key system sources to produce short and long-term outcome analyses. Key data sources include but are not limited to case file reviews, program narratives, program observations, surveys, and administrative data on enrollment, attendance, invoicing, activities and outcomes.

3) Prepare and submit quarterly MOE data reports for internal and public audiences.

4) Generate graphs, tables, and other data visuals.

5) Assist in survey design, administration, and analysis.

6) Automate regular data reporting of administrative, compliance, and quality data.

7) Monitor contract compliance by coding and maintaining provider-specific automated monthly reports.

8) Craft novel, exploratory reports of learning, achievement, and demographic data to support the conception and articulation of MOE's network-wide strategies.

9) Support Adult Education (AEPHL) Continuous Quality Improvement (CQ) process.

a. Monitor and support the collection Of program evaluation data using CQI tools across the AEPHL network of adult education providers.

b. Clean, analyze, and disseminate provider-level CQI data for AEPHL network partners.

c. Analyze CQ and evaluation data in aggregate to communicate trends in program improvement to all MOE and AEPHL stakeholders.

10) Support the strategic planning, facilitation, and day-to-day management of the Continuous Program Improvement Collaborative (CPIC), a yearlong professional development initiative that equips adult education providers to collect, monitor, and analyze data to inform continuous program improvement and decision-making.

11) Ensure transparency and accountability in service provider communications through consistent external reporting and the solicitation of provider stakeholder input and feedback.

12) Advise teammates on measurement and evaluation processes, including formulating research questions, planning the scope of evaluation, and developing feedback cycles between data collection and data analysis.

13) Develop written reports and presentations for a variety of audiences using clear, concise language.

14) Perform related research and evaluation duties as assigned.

15) Work collaboratively with PMT team members to ensure quality implementation of performance management activities and processes.

16) Ensure data project requirements are achieved in a timely fashion.

17) Other duties as assigned.

Competencies, Knowledge, Skills and Abilities

Quantitative:

Knowledge of statistical and survey software such as R, STATA, SPSS, and Qualtrics.

Knowledge of basic statistics including descriptive, exploratory, and inferential statistics.

Prior experience creating basic data visualizations such as charts and tables.

Knowledge of survey design, programming, and administration.

General:

Knowledge of program evaluation and outcomes measurement.

Strong research, analysis, and writing skills. Clear and concise communication of ideas, program scopes, and findings — orally and in writing.

Ability to problem solve and identify several courses of action to make informed decisions about project development and implementation.

Ability to maintain strong working relationships with coworkers, leadership, AEPHL staff, and service providers.

Ability to work on multiple projects at once while still meeting project deadlines.

Ability to quickly pivot to emergent tasks and project needs.

Knowledge of Microsoft Office Suite, including data analysis tools like Excel and/or Open-Source tools.

Demonstrate knowledge of and experience with mixed methods research design and data analyses.

Demonstrate experience with leading multiple, simultaneous data projects and with delivering such projects by the agreed deadline.

Know personal strengths, weaknesses, opportunities, and limits.

Reflective and seeks feedback.

Exceptional cultural competency and proficiency.

Qualifications

Qualitative:

Ability to support evaluation and monitoring tool development, including conducting literature reviews for existing tools and using stakeholder input to update existing tools

Prior experience conducting interviews, focus groups, and observations

Prior experience analyzing qualitative data, including coding interviews, focus groups, and observations and conducting document reviews

Quantitative:

Knowledge of statistical and survey software such as R, STATA, or python

Knowledge of basic statistics including descriptive, exploratory, and inferential statistics.

Prior experience creating basic data visualizations such as charts and tables.

Knowledge of survey design, programming, and administration including platforms such as Qualtrics or Cognito

General:

Knowledge of program evaluation and outcomes measurement

Strong research, analysis, and writing skills

Clear and concise communication of ideas, program scopes, and findings — orally and in writing Ability to problem solve and identify several courses of action to make informed decisions about project development and implementation

Ability to maintain strong working relationships with coworkers, OCF leadership, and service providers

Ability to work on multiple projects at once while still meeting project deadlines

Ability to quickly pivot to emergent tasks and project needs

Knowledge of Microsoft Office Suite, including data analysis tools like Excel and/or Open Source tools

Experience with shortand long-term research design and analysis and the delivery of research products for lay/practitioner audiences.

• Bachelor's Degree in Social Policy, Government Administration, and/or other relevant Social Science
• At least 2 years of data analytics and cleaning, programming, research and evaluation design, and