SLC Evaluation Simulation

Overview of the Evaluation Plan

The evaluation of the PLC program at Westover High School will be objective-based. The evaluation team has determined a set of objectives that will be evaluated to determine if the PLC program should continue. The evaluation will focus on aspects of the program that will determine if the objectives are being met.

Detailed Evaluation Plan

Clarify the Evaluation Request and Responsibilities

The School Board has requested that the Superintendent, by the end of the next school year, report on the success of the Small Learning Community program at Westover High School. The evaluation will be used by the School Board to determine if the program is successful and if it should continue. The evaluation will look at [GHO1]student achievement, program promotion, student transition, discipline referrals, dropout rates, attendance rates, teacher and student surveys and allocation of resourcesto determine if the program should continue[GHO2].

Analysis of the Evaluation Context, Limiting the Study

The largest factor limiting the study is the lack of objectives for the program[GHO3]. The program does not have stated objectives and goals. One of the first steps of the evaluation will be the development of stated objectives for the Small Learning Community program at Westover High School. The principal and the PLC faculty will be responsible for developing objectives of the program. The evaluation will be an objective-based evaluation based on the objectives they determine for the program. The evaluation will look at student achievement, program promotion, student transition, discipline referrals, dropout rates, attendance rates, teacher and student surveys and allocation of resources to determine if the program is meeting its objectives.

Identification of Intended Audiences to Receive the Evaluation Results

The intended audience of the evaluation results is primarily the School Board. The Superintendent will be responsible for analyzing [GHO4]the data that will be reported to the School Board. The School Board may then elect to share the results with the public as a matter of public information.

Describe what is to Evaluated

After consulting with the faculty and staff of the Small Learning Community program, the principal of Westover High School determines the following objectives for the program.

-The SLC program will increase student achievement.

-The SLC program will provide easy transition from middle school to high school by providing focused attention and guidance to students throughout the transition.

-The SLC program will contribute to a reduction in discipline referrals.

-The SLC program will contribute to a decrease in dropout rates and absences.

The evaluation will measure the success of the program meeting its objectives by evaluating the following aspects of the program; student achievement, program promotion, student transition, discipline referrals, drop-out rates, attendance rates, teacher and student surveys[GHO5] and allocation of resources.

Analyze Resources and Capabilities that can be Committed to the Evaluation

Due to the current economic situation and the school system budget, the evaluation will be a cost-free evaluation, using the human resources of the Small Learning Community at Westover High School.

The Superintendent has appointed an internal evaluation team consisting of the following; the Westover High School Principal, one Westover High School Assistant Principal, the Guidance Counselor involved with the program, a Guidance Counselor not involved with the program, one teacher representative from each of the four cluster areas, one teacher representative not involved with the program, a testing coordinator and one volunteer parent of a student in the program.[GHO6]

Each member of the evaluation team will be trained in the objectives of the program and will be assigned a specific aspect of the program to evaluate with specific criteria. The members of the team will be encouraged to provide unbiased data.

Analyze the Political Context for the Evaluation

There is a political aspect to every evaluation. In this simulation, the group of people with the most to gain or loose, are the students in the Small Learning Community program. The decisions made will affect the students, followed by the faculty and staff involved with the program. The school principal and the Superintendent have some level of power in this situation. The School Board has the ultimate power in this situation. They will be determining the future of this program. Cooperation from all stakeholders will be important to provide unbiased and accurate information for the evaluation. Those stakeholders include; administrators, teachers, staff, students, parents, volunteers and community members.

Questions Posed by the Evaluation

The initial question posed by the School Board is to evaluate how successful the Small Learning Community program is[GHO7]. The Superintendent has left the details of the evaluation up to the evaluation team. The principal and the faculty and staff of the PLC program determined the objectives of the program. [GHO8]The evaluation team came up with a long list of the aspects of the program to be evaluated. These tasks refer to the divergent phase of identifying and selecting evaluation questions.

In the convergent phase the final questions/areas were selected from the proposed questions determined during the divergent phase. Cronbach suggests several criteria for determining which proposed evaluation questions should be investigated. The following matrix (figure 13.2- Fitzpatrick, Sanders, Worthen) was used to determine which areas of the PLC programshould[GHO9] be investigated and which do not play a crucial role in the outcome of the evaluation.

Would the evaluation question…Evaluation Question

1 2 3 4 5 6 7 8 9 10 [GHO10]

  1. Be of interest to key audiences?
  2. Reduce present uncertainty?
  3. Yield important information?
  4. Be of continuing (not fleeting) interest?
  5. Be critical to the study’s scope and comprehensiveness?
  6. Have an impact on the course of events?
  7. Be answerable in terms of
  8. Financial and human resources?
  9. Time?
  10. Available methods and technology?

After applying this matrix to the list of initial questions, the team proposed looking at the following areas of the PLC program; student achievement, program promotion, student transition, discipline referrals, drop-out rates, attendance rates, teacher and student surveys and allocation of resources.

Data Collection Procedures

Each team member will be assigned an aspect of the evaluation to collect data and analyze the results. The results will be reported back to the team and compiled into a final report. All of the results will be analyzed and summarized into a final report showing the details and summarizing the success of the program. Some of the questions will be causal to determine causality and others will be descriptive in nature. The descriptive questions will look for trends, processes, or try to describe and analyze the pros and cons of the PLC program.

Identify Information Needs and Variables and Their Sources

To collect the necessary data, appropriate sources of information must be determined for each question[GHO11]. The source of information can be particular individuals or existing information. Relevant data can also be collected from test results, schools records and other evaluations.

Identify Tests, Instruments, and Other Specific Data Collection Methods

The following aspects of the SLC program will be evaluated using these sources[GHO12]:

Student achievement:

-Compare EOC scores of non-SLC students and SLC students

-Compare GPA scores of non-SLC students and SLC students

Program promotion:

-Evaluate how the program is advertised and promoted to middle school students, parents and teachers

-Survey students, parents and teachers about the promotion of the program

-Was the program promoted effectively?

-If not, what information was lacking?

-How can the program be better promoted?

Student transition:

-Survey non-SLC students and SLC students on aspects of the transition from middle school to high school to determine if one group had a smoother transition.

-Are there any differences in student opinions?

-Is one group having an easier transition than the other?

Discipline referrals:

-Compare the number of discipline referrals of non-SLC students and SLC students within the same year

-Compare the number of discipline referrals of students in upper grades compared to previous years

-Does the SLC program improve discipline after they leave the program and become upperclassmen?

Drop-out rates:

-Compare the number of dropouts after 9th grade for non-SLC students and SLC students for this year.

-Compare dropout rates for this year compared to last year before the program was in place.

-Does the PLC program affectdropout rates?

Attendance rates:

-Compare the attendance rates of non-SLC students and SLC students for this year.

-Compare the attendance rates for this year and last year.

Teacher, Student and Parent Surveys:

-Opinion surveys about the SLC program from teachers, students and parents.

-Opinion surveys from non-SLC students about the school.

Allocation of resources:

-Are resources being evenly distributed to the SLC program and the non-SLC aspects of the school?

-Financial reports/budgets.

Describe the Conditions Under Which Information Will be Collected

Due to the relatively small size of the PLC program, random sampling will not be necessary for this evaluation. Every attempt will be made to collect descriptive, opinion and survey data from every member of the program. Test results and statistical data will be available for all students in their school records. Student names will not be used in any reports to protect student identity.

All opinion surveys will be given by the specified member of the team assigned to that area of the evaluation. Another member of the team not responsible for that area of the evaluation will be present while the survey is being administered and will assist in compiling and verifying the results. Students and parents will be told the purpose of the survey and the evaluation. The importance of their opinions and honest feedback will be conveyed to all people taking surveys. All surveys will be anonymous, so there is no fear of retaliation from administrators or teachers. The surveys will be administered near the end of the school year after the students; parents and teachers have had one full academic year to experience the PLC program. The other data will be collected as test results are revealed.

Describe How the Data and Information Are to be Organized and Analyzed[GHO13]

Each member of the team will be responsible for organizing the data into databases and providing a visual chart or diagram summarizing the results of their particular area of the evaluation. Opinion surveys should show the percentages of people that agree or disagree with a particular statement and/or show the percentage of satisfaction with a certain area of the program. Test results and attendance rates should show a percentage comparing PLC students and non-PLC students. The results will be interpreted by the team and other stakeholders to elicit multiple interpretations of the results, before a final interpretation is made.

The worksheet for Summarizing an Evaluation Plan in Figure 14.1 (Fitzpatrick, Sanders, Worthen) would be used by each team member to stay organized and focused while collecting data. This worksheet will be included in the final report to show how each questions was evaluated:

Evaluation Plan Summary

Evaluation Questions or Objectives / Information Required / Information Source / Method for Collecting Information- by whom, conditions, when / Analysis Procedures / Interpretation Procedures and Criteria /
Reporting of Information- to whom, how, when
Student achievement / Test scores / Test scores / Compile test scores / Described below / Described below / Described in Gantt Chart below
Program promotion / Descriptive Promotion Plan
Surveys / Promotion Plan
Surveys / Descriptive
Surveys
Student transition / Surveys / Surveys / Surveys
Discipline referrals / Office Referral Data / Office Records / Compile data
Drop-out rates / Drop-out data / Office records / Compile data
Attendance rates / Attendance records / Office records / Compile data
Teacher, Student and Parent Surveys / Surveys / Students, Parents and Teachers / Surveys
Compile data
Allocation of resources / Budgets
Financial Statements / Office records / Compile data

Management Plan for Conducting the PLC Evaluation

MS= Milestone- product/report compiled to show completion

Week 1 / Week 2 / Week 3 / Week 4 / Week 5 / Week 6 / Week 7
Develop
Measures/Objectives / X
MS
Conduct
Observations / X
Administer
Surveys / X
Conduct
Interviews / X
Analyze Data / X
MS
Review with
Stakeholders / X
Prepare Final Report / X
MS
Present to
School Board / X

Personnel Requirements

The Superintendent has appointed an internal evaluation team consisting of the following; the Westover High School Principal, one Westover High School Assistant Principal, the Guidance Counselor involved with the program, a Guidance Counselor not involved with the program, one teacher representative from each of the four cluster areas, one teacher representative not involved with the program, a testing coordinator and one volunteer parent of a student in the program.[GHO14]

Personnel Assignments

Evaluation Area: / Personnel Responsible:
Student achievement / Testing Coordinator/PLC Teacher
Program promotion / Parent Volunteer
Student transition / PLC Teacher/non-PLC Teacher
Discipline referrals / Assistant Principal
Drop-out rates / PLC Teacher
Attendance rates / PLC Teacher
Teacher, Student and Parent Surveys / PLC Guidance Counselor/non-PLC GC
Allocation of resources / Principal

Budget

Due to the current economic situation and the school system budget, the evaluation will be a cost-free evaluation, using the human resources of the Small Learning Community at Westover High School and other school personnel. The evaluation will also use other school resources such as computers, paper, and office supplies.

[GHO1]Wording. I think you mean something like “examine,” or “evaluate,” or anything except “look at.”

[GHO2]Evaluators do not make these decisions…stakeholders do. What the evaluation will do is provide information to stakeholders (the Board) so that they can make an informed decisopm.

[GHO3]Actually, all programs have goals and objectives. It is up to you to articulate these. Presumably, Westover started the SLC program after some research on SLCs. You should do the same thing. The literature on SLCs has much to offer about the benefits provided by SLCs, how they should be organized, what kinds of problems to watch out for, tec. You need to investigate that literature.

[GHO4]You are passing the buck. The evaluator will analyze the data and report the results to the superintendent who, in turn, will interpret the results for the school board.

[GHO5]What are you going to survey? What will you be attempting to learn from your surveys?

[GHO6]You don’t mean that all these individuals are on the evaluation team, do you. While these individuals may be called upon to provide data, surely they are not all “on the team?”

[GHO7]This is NOT a question.

[GHO8]As I stated in an earlier comment, there would have been a set of goals and objectives for the program before it got implemented. You can assume that thesecan be assumed to be congruent with the typical goals and objectives of SLCs elsewhere. Your job it to find out what those goals and objectives are.

[GHO9]…to…

[GHO10]What is this?

[GHO11]So?...what are these?

[GHO12]What sources? Good set of questions, though.

[GHO13]You have had RES 5000 (Research Methods). Your should be able to use some of what you learned there for this evaluaton.

[GHO14]Again, the evaluation team, for a project like this, would never be so large.