Evaluation Plan: Fountas & Pinnell Benchmark Assessment System

EVALUATION PLAN 2

Evaluation Plan: Fountas & Pinnell Benchmark Assessment System

Kimberly A. Sharp

University of West Georgia

Evaluation Plan: Fountas & Pinnell Benchmark Assessment System

Introduction

Background Information

Ivy Creek Elementary School is located in northeastern Gwinnett County in Buford, Georgia. Founded in 2003, the school is comprised of 886 students in grades PreK-Fifth Grade. At a recent faculty meeting, the principal of Ivy Creek, Yvonne Frey, shared statistics about the students attending the school. Demographically, the student population is 58% Caucasian, 20% Hispanic, 10% African – American, 8% Asian and 4% Multi-Racial. Approximately 29% of students receive free or reduced lunch while the majority of students come from families with average socioeconomic status (Y. Frey, personal communication, May 17, 2010).

Gwinnett County prides itself on becoming a system of world-class schools. As a member of the largest school district in the state of Georgia, Ivy Creek Elementary has enjoyed the support of an active Language Arts Department. The school was recently invited to pilot the Fountas & Pinnell Benchmark Assessment System largely as a result of the efforts of the reading coach at Ivy Creek Elementary School.

The administrative staff at Ivy Creek Elementary School requested that this program be evaluated to determine the impact of the assessment system on reading instruction and student achievement in reading. This evaluation will provide valuable information to the Ivy Creek Elementary School Stakeholders: Principal, Assistant Principals, Reading Coach, Teachers and Staff. In addition, the results of the evaluation will be available for review by the county office staff. This data may have an impact on whether the program is recommended to all elementary schools in the county. As the media specialist at Ivy Creek Elementary School and one of the few teachers not directly involved with administering the Fountas & Pinnell Benchmark Assessment System, I am conducting an objective evaluation of the program.

Evaluation Program

Fountas and Pinnell have spent more than twenty years establishing a leveling system for educators to use when evaluating students’ reading ability. Teachers are able to determine whether a student is meeting the standards set for each grade level by using leveled books and measuring a student’s reading fluency and comprehension. Levels follow a twenty-six point scale from the easiest level (A) to the most challenging level (Z.) These alphabetical levels correspond with reading ability levels ranging from Kindergarten to 8th Grade (Fountas & Pinnell, 2008.)

The Fountas & Pinnell Benchmark Assessment Program is designed to measure student’s reading levels with the use of benchmark books. These leveled books are used during a one-on-one session between the teacher and the student. During these sessions, the student reads the book aloud and discusses the book while the teacher observes and codes the behavior of the reader. This system of coding allows the teacher to identify student errors with regards to fluency and comprehension. The information gained from this personalized assessment session provides the teacher with valuable insight into the reading ability of the student and the concepts that need to be addressed during reading instruction (Fountas & Pinnell, 2008.)

Purpose

The purpose of conducting an evaluation of the Fountas & Pinnell Benchmark Assessment System at Ivy Creek Elementary School is to determine the impact it has on reading instruction and student achievement. The program is designed so that teachers administering the assessment are able to use student errors to guide instruction. As a result, reading instruction is more focused and allows students to progress more rapidly through the stages of reading.

This assessment system is closely aligned with the district’s views of reading instruction. Specifically, the program correlates with the use of guided reading and reading workshops as methods of delivery. In a program study of a reading initiative designed to improve reading achievement, Al Otaiba and Hosp recommended that assessment programs should align with district reading programs (2008.) The Language Arts Department feels confident that the Fountas & Pinnell Benchmark Assessment Program will meet the needs of reading teachers.

Unfortunately, complete student achievement data will be unavailable during the time constraints for this project. While there will be some indication of student progression in reading, official midterm and year-end assessments will not be administered until January and again in May. Therefore, the focus of this project will be on professional learning related to implementing the assessment system and the impact this program is having on teacher instruction.

At the completion of this formative evaluation project, the leaders at Ivy Creek Elementary School will have a true picture of how the Fountas & Pinnell Benchmark Assessment System is impacting classroom teachers and their reading instruction. Additional professional learning opportunities will be offered to teachers based on the needs outlined in this evaluation. This training, if needed, will be provided to teachers prior to the administration of the midterm and year-end assessments. The data collected during the course of this project will also provide professional learning instructors with specific feedback on how to further guide teachers in administering effective reading instruction.

These steps re-enforce the ideas suggested by Engstrom and Danielson in their review of a professional development program on Howard Gardner’s multiple intelligence theory. They found that teachers need ongoing support when implementing a new program. Additionally, they recommended that action research be conducted throughout program implementation so that professional development could be focused on the needs of teachers (2006.) It is also critical for teachers to have sufficient time and resources to understand the intent of new standards and the structure of new curriculum (Obara & Sloan, 2009.)

Once the needs of teachers have been identified, professional development opportunities can be designed to address those needs. It will be important to narrow the focus of these sessions to the most relevant topics while integrating specific content and teaching ideas. It will also be necessary to provide teachers with networking time and long-term support (Hutchins, Arbaugh, Abell, Marra & Lee, 2008) to ensure that the implementation of the Fountas Pinnell Benchmark Assessment Program is done correctly.

After teachers have administered the mid-term and post-evaluations, the evaluator and the client plan to review student reading achievement data; CRCT scores, Guided Reading Levels, and Reading Standards. This hard data should serve to reinforce the qualitative data obtained as a result of this evaluation project (Kaufman, Guerra, & Platt, 2006.) Although this data will be obtained beyond the scope of this evaluation, it will be necessary to include student achievement results in order to complete the full program evaluation. This summative evaluation is scheduled for completion by the end of the 2010-11 school year.

Evaluation Questions

In order to provide meaningful information to Ivy Creek Elementary School stakeholders through this evaluation, the following questions must be answered:

·  What impact do professional learning sessions have on teachers as they implement the Fountas & Pinnell Benchmark Assessment System?

·  How does the Fountas & Pinnell Benchmark Assessment System impact reading instruction?

·  How does the Fountas & Pinnell Benchmark Assessment System compare to the assessment program used last year?

·  Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at Ivy Creek Elementary School next year?

Methods

Participants

All certified staff members who administer the Fountas & Pinnell Benchmark Assessment System will be asked to participate in this study. This group will include classroom teachers in Grades 1-5, the reading coach, intervention specialists and gifted reading teachers.

Design and Procedures

A variety of activities will be designed to address all of the objectives outlined in this evaluation plan. A quantitative research approach will be used in that data will be gathered from teacher surveys (Appendix A) and interviews (Appendix B). Questions have been designed to evaluate teachers’ perceptions of their professional learning experience as related to the Fountas & Pinnell Benchmark Assessment System. Additional questions address the impact the program has had on the way teachers delivered reading instruction. Comparing the Fountas & Pinnell Benchmark Assessment System to the reading programs used in years past will be an important factor in determining whether or not to proceed with the adoption of the Fountas & Pinnell model. Qualitative data will be gathered from open-ended and closed-ended questions in both surveys and in-person interviews.

Instruments

A comprehensive survey has been developed by the evaluator for approval by the client. Since only one survey will be administered for the purpose of this project, a select group of three teachers will be asked to review the survey for clarity. All certified staff members responsible for reading instruction will be asked to complete the survey. Questions will be both open and closed-ended.

Once the surveys are complete, in-person interviews will be conducted with teachers who are willing to participate. Ideally, at least one member from each grade level will be interviewed using the interview guide approach (Appendix B.) This qualitative data collection method will allow for topics and issues to be specified in advance while allowing for variance in the sequence and wording of the questions as the interview progresses. This method will keep the interview conversational which perhaps will elicit more meaningful data (Johnson & Christensen, 2007.)

Finally, informal student assessment data will be used to determine if students appear to be progressing in the area of reading achievement. Since the formal assessment will not be administered before the end of this project, the evaluator will examine student work and rely on teacher observation to determine if students are showing signs of achievement.

Objective / Survey / Interview / Informal Student Assessment
What impact do professional learning sessions have on teachers as they implement the Fountas & Pinnell Benchmark Assessment System? / x / x
How does the Fountas & Pinnell Benchmark Assessment System impact reading instruction? / x / x
How does the Fountas & Pinnell Benchmark Assessment System compare to the assessment program used last year? / x / x / x
Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at Ivy Creek Elementary School next year? / x / x / x

Data Analysis

Surveys

Each question on the survey will be carefully analyzed to determine patterns of response by the participants. Those questions that are answered using a Likert scale will be averaged to determine an average score for the respondents. These responses will be analyzed further based on age, teaching experience and educational degree to determine if those factors are relevant. Open-ended responses will be grouped based on similarities.

Interviews

Ideally, at least five teachers from various grade levels will volunteer to participate in the interview process. The interviews will be recorded and analyzed to determine patterns of response by the participants. Anecdotal commentary will be included in the evaluation report.

Informal Student Assessment

As a part of each interview, the evaluator will review the class reading level data provided by the teacher. Although this will be a work in progress, a majority of students in each class should show growth in reading level as compared to their initial assessment..

Conclusion

Upon completion of the data analysis, the evaluator will prepare a written draft of the evaluation report. This draft will be shared with the reading coach to review the findings and determine any areas that need clarification. As part of this process, the reading coach will also be asked to complete a language impact questionnaire (Appendix C) as recommended by Kaufman, Guerra and Platt (2006.) Based on the findings of the language impact questionnaire, the evaluator will revise the written evaluation report for presentation to the administration at Ivy Creek Elementary School.

During the executive briefing, the evaluator and client will discuss the findings of the formative program evaluation. Plans for modifying professional development and current teaching practices will be discussed as necessary based on the results of the evaluation. Additionally, a plan for incorporating mid-term and post-assessment achievement data in the coming months will be developed in order to complete the summative evaluation. At the end of the school year, all final results of the evaluation program will be shared publicly with the staff.

References

Al Otaiba, S., Hosp, J., Smartt, S., & Dole, J. (2008). The challenging role of a reading coach, a cautionary tale. Journal of Educational & Psychological Consultation, 18(2), 124-155. Retrieved from ERIC database.

Engstrom, M., & Danielson, L. (2006). Teachers' perceptions of an on-site staff development model. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 79(4), 170-173. Retrieved from ERIC database.

Fountas, I. & Pinnell, G. (2008). Assessment guide: A guide to the benchmark assessment system. Portsmouth: Heinemann.

Hutchins, K., Arbaugh, F., Abell, S., Marra, R., & Lee, M. (2008). A consumer guide to professional development. Science Scope, 31(8), 16-19. Retrieved from ERIC database.

Johnson, B., & Christensen, L. (2007). Educational research: Quantitative, qualitative, and mixed approaches (3rd ed.). Boston: Allyn and Bacon.

Kaufman, R., Guerra, I., Platt, W. (2006). Practical evaluation for educators: finding what works and what doesn’t. Thousand Oaks: Corwin Press.

Obara, S., & Sloan, M. (2010). Classroom experiences with new curriculum materials during the implementation of performance standards in mathematics: A case study of teachers coping with change. International Journal of Science and Mathematics Education, 8(2), 349-372. Retrieved from ERIC database.

Pinnell, G. & Fountas, I. (2007). The continuum of literacy learning: A guide to teaching. Portsmouth: Heinemann.

Purdue University Online Writing Lab (OWL) (May 5, 2010). APA formatting and style guide. Retrieved from http://owl.english.purdue.edu/owl/resource/560/01/.

Appendix A

Survey – Fountas & Pinnell Benchmark Assessment Program

The following survey is designed to evaluate the Fountas & Pinnell Benchmark Assessment Program (Fountas & Pinnell Program.) It should take approximately 5-10 minutes to complete. Your answers will provide valuable information in determining the effectiveness of the program as it relates to reading instruction and professional development. No names will be used at any time when reporting data for this evaluation, so please answer honestly!