Reach Out Iowa

Final Project Evaluation Summary

Prepared by the

Research Institute for Studies in Education

Iowa State University

August 2012

INTRODUCTION

The Research Institute for Studies in Education (RISE), Iowa State University served as external evaluator for the “Reach Out Iowa” (ROI) project funded by the Corporation for National and Community Service under its Learn and Serve America initiative. This report presents a final summary of evaluation activities and findingsof the ROI project. During Year One RISE finalized the project evaluation plan, developed the youth participant and recipient organization surveys, and developed on-line instruments for ROI staffto collect project and demographic survey data. Evaluation activities during Years Two and Three included (1)evaluation of the training provided for leaders of service learning projects, (2) data collection from youth who participated in ROI service learning activities, (3) data collection from ROI recipient organizations,(4) focus groups and interviews with youth and adults who had been involved in ROI service learning projects, and (5) a survey of project leaders about partnering with Iowa State University Extension (ISUE). A summary of the evaluation activities and their findings follows. Complete details of the evaluation can be found in interim and annual reports provided to project staff.

Evaluation of Service Learning Training

ISUE conducted 4-H Youth Development Volunteer Training in all 99 counties throughout the state in Year Two of the project. After the training was completed, participants were asked to complete an evaluation instrument designed and distributed by the trainers that included: level of agreement with statements about their knowledge and skills for leading service learning projects, recommendation of the training for other volunteers, and suggestions for improving the training and/or future training topics. RISE received and analyzed results from 343 respondents representing 27 counties.

This training was very effective in preparing volunteers to apply the service learning approach (IPARDC process) to ROI community projects. Most of the volunteer leaders understand service learning, the process, and resources available for carrying out projects. They are confident in their ability to use their training and adopt the framework for future service projects. Almost all participants would recommend the training to other volunteers.

Youth Participant Survey

During Years Two and Three of the project youth participants were asked to complete a brief survey after completing their service learning project. The instrument was developed by the evaluation team and project staff and was available in both paper and online versions. Part I asked youth to indicate whether statements were definitely or mostly false or mostly or definitely true in describing themselves before and after their service learning experience. Statements were related to civic knowledge, leadership, community attachment, community responsibility, and perceptions of personal skills. Part II asked about their perceptions of the experience and its effects on them. RISE received and analyzed 2093 surveys representing 101 projects.

Youth participantsexperienced significant changes as a result of their participation in service learning projects over the course of the ROI project. Civic knowledge, leadership, community attachment, community responsibility, and perceptions of personal skills were greater after the experience as compared to before. There was little difference in youth responses from Year Two to Year Three of the project, with mostyouth stating they benefitted for working with others and felt included and valued in the projects. Youth respondents consistently stated that their ROI experiences made them care more about needs in their communities and they plan to do other service or volunteer projects. In both Years Two and Three the project met a key performance measure that at least 50 percent of youth would increase their civic knowledge.

Recipient Organization Survey

During Years Two and Three of the ROI project, leaders of organizations that were recipients of services from the youth projects were asked to complete a brief survey about project planning and implementation and their perceptions of the youth participants. The instrument was developed by the evaluation team and ROI project leaders and was available in both paper and online versions. RISE received and analyzed surveys from 107 respondents representing 49 projects.

Overall, recipient organizations were extremely positive about the planning and implementation process. They consistently indicated that the service learning projects were beneficial to their clients/recipients and that they, too, benefitted from their interaction with the youth and are now more interested in working with youth in the future. In both Year Two and Year Three of the project, a key performance measure was met that at least 75 percent of recipient organizations would report that the clients/recipients are benefitting from and/or finding the activities provided by the service learning participants to be useful.

Youth and Adult Focus Groups and Interviews

RISE conducted eleven focus groups and three interviews assessing the experiences of adults and youth involved with Reach Out Iowa during Years Two and Three of the project. The focus groups and interviews were conducted at project locations across Iowaduring May and June of 2011 and 2012. Most locations held both a youth focus group and an adult focus group;some held a youth focus group and/or an adult interview. The focus groups and interviews covered the following topics:

  • The IPARDC Process
  • Roles and responsibilities of adults and youth in ROI projects
  • Accomplishments and benefits of ROI projects
  • Differences between ROI and other service learning projects
  • Strengths and challenges of working with ISU extension
  • Sustainability of ROI projects

Overall, group members were very pleased with their ROI projects, experiences, and results. Themes that emerged over Years Two and Three of the project included:

Successes:

  • Youth experienced increased positive outcomes of the ROI projects; they felt proud of their work and reported an eagerness to continue to be engaged in their communities.
  • For most projects, youth were successful in completing the bulk of ROI projects themselves.
  • Projects created a sense of community responsibility in youth and had a positive impact on the community and community residents and created community partnerships,
  • Adults in the community, including teachers and project leaders, began to see youth as capable, responsible, and generous when provided the opportunity.
  • Youth learned communication, teamwork, leadership, and other practical skills.
  • Youth enjoyed their experiences with Reach Out Iowa and experienced a sense of accomplishment.
  • Participants felt that ROI projects were significantly different than other project they had been involved with in the past, and adults and youth alike expressed the desire to continue to engage in service learning projects and felt that they would be able to do so without Reach Out Iowa funding.
  • ISU extension staff provided a useful curriculum and was helpful and supportive.

Challenges:

  • Youth who were not as engaged in all phases of the IPARDC process experienced less positive outcomes; they did not feel as proud of their work, they did not express desires to continue to engage in service learning, they reported learning less, and they did not report feeling an increased connectedness to their communities.
  • Adults maintained major roles in a couple of the projects due to short timelines.
  • Some adults in the community, including teachers and project leaders, were resistant to seeing teenagers as capable, responsible, and generous people.
  • Some adults found the paperwork and reimbursement processes associated with the grant to be burdensome.
  • Certain aspects of the IPARDC process were not fully utilized; for instance, few adults and youth reported celebrating their accomplishments at the end of the project and a few students did not know what focus group facilitators meant when asked about “reflection.”

Extension Partnership Survey

In order to ensure that all project leaders had an opportunity to comment on their experiences working with Iowa State University Extension partners, RISE prepared a brief online survey to gather feedback about their perceptions of ISUE as a partner and the likelihood they would continue using service learning as a tool in their organizations. Project staff emailed an invitation with the survey link to 39 potential respondents. RISE received and analyzed the nine responses. Eight of the nine said they had worked directly with a youth planning team. Five had served as ISUE’s primary contact for the project. Four had helped with project logistics or completed paperwork.

Leaders’ experienceswith ISUEwere very positive, especially regarding communication, ISUE’s response to questions, and materials provided for youth training. Most thought ISUE provided valuable suggestions and was important to the success of the projects. Over three fourths are likely to continue using service learning in the future.

Overall Summary

Overall, the Reach Out Iowa project was very successful in providing leadership and support for both adult volunteers and youth participants in service learning projects. The IPARDC process has been consistently implemented with fidelity. In most cases, youth have assumed primary roles in the investigation, planning, and actionphases. The demonstration and celebration phasesare being implemented, but youth aren’t necessarily playing as great a role as the adults in these areas. The reflection phase is being implemented, but youth aren’t necessarily aware that it is occurring or familiar with the term for what they are doing.

ROI project staff and Iowa State University Extension have been excellent partners with service learning project leaders throughout the project. Leaders commended ISUE staff for their communication, responsiveness, and supporting materials, although a few noted that reimbursement procedures were slow and reporting requirements burdensome.

Youth learned a great deal about themselves and their communities through planning and carrying out ROI service learning projects. As a result of their involvement, they are more aware of and care more about needs in their communities and plan to be involved in future volunteer or service projects. Adult leaders found the IPARDC service learning model to be effective. Leaders from recipient organizations learned that youth are more capable and caring than they thought, benefitted from their interactions with the youth, and intend to work with them on future projects. Key performance measure targets for increasing the civic knowledge of youth and meeting needs of recipients were met.

Prepared by the Research Institute for Studies in Education (RISE), Iowa State University1