Evaluation Plan
FY2014-2015
Tool/Measure
/Purpose
/When
/Who Completes (or Retrieves)
/Where to Access
/Reports to Use
/Implementation Fidelity – Completed with the help of your school wide facilitator (SWF)
Self-Assessment Survey (SAS) and School Safety Survey (SSS) / Identify status, needs and change over time / Prior to initial 3 day SWPBS training, then Annually / All building level staff complete and review; Anonymous data are retrieved by PA PBS evaluators via secure website / www.pbisapps.org / Individual Item Scores: Current status (in place, partially in place, and not in place) and improvement priority (high, medium, and low) percentages for each item within each system.Team Implementation Checklist (TIC) / Implementation Status and Technical Assistance Needed / For schools in Universal Phase I and II: October, December, March / Team completes checklist and comes to consensus. SWF enters on PBIS Assessment. / www.pbisapps.org / Implementation by Feature: This report shows the percentage implemented and partially implementedfor each feature.
Benchmarks of Quality (BoQ) / Implementation status and change over time / Annually / Building Coach and Team Members. District Coach submits final BoQ to SWF who then enters on PBS Surveys website; PA PBS evaluators retrieve via secure website / www.pbisapps.org / Elements Report: Percent implemented for each BoQ element in a bar chart and accompanying data table.
School-wide Evaluation Tool (SET) (by appointment after achieving a 70 or higher on the BoQ) / Assess and evaluate the critical features of school-wide positive behavior support / Minimally every 3-5 years based on need for additional data and/or recognition purposes / Trained external evaluator completes and reviews with team; PA PBS evaluators retrieve via secure website
(by appointment after achieving a 70 or higher on the BoQ) / www.pbisapps.org / SET Summary Score Report: Used (a) to determine annual goals for SWPBS (b) to evaluate on-going efforts (c) design and revise procedures as needed, (d) to compare annual accomplishments
Outcomes – data gathered in summer for program evaluator with help from your SWF and school point of contact
School Safety Survey (SSS)
/Identify safety status, needs and change over time
/Annually
/All building level staff complete and review; Anonymous data are retrieved by PA PBS evaluators via secure website
/ www.pbisapps.org /Office Discipline Referrals (ODR)
/ Evaluate impact on rates of problem behavior / Monthly (by team), Quarterly (shared with staff), Annually shared with PAPBS evaluators / Enter in SWIS or other data system; If schools use SWIS, their annual SWIS data are automatically accessed by PAPBS evaluators via secure website / www.pbisapps.orgSchool district and building level database
/“Big Five Reports”:
· Avg Referrals per Day per Month
· Referrals by Problem Behavior
· Referrals by Location
· Referrals by Time
· Referrals by Student
Pennsylvania School System of Assessment (PSSA) / Evaluate impact on academic performance / Annually / PAPBS evaluators retrieve via public access / http://paayp.emetric.net/Pennsylvania Value Added Assessment System (PVAAS) / Evaluate impact on annual academic growth / Annually / PAPBS evaluators retrieve via public access / https://pvaas.sas.com/evaas/welcome.jsf
Safe Schools / Evaluate impact on violence and weapons possession on school property / Annually / PAPBS evaluators retrieve via public access / www.safeschools.state.pa.us
School Profile
· Total Students
· Student attendance rate
· Staff attendance rate
· LRE per 3 reporting categories
· Referrals for special education eligibility determination
· Newly identified special education identifications
· OSS/ISS
· Out-of-school placements for students
· Out-of-school placements for students with ED
· Collaborating MH agency
· Number of students with IEPs / Used to identify changes over time with implementation / Annually / School/District Point of Contact (POC) submit to PAPBS evaluators / www.papbs.org
Data Collection Schedule
Fall / Winter / Spring / Summer / Fall / Winter / Spring / Summer / Fall / Winter / Spring / Summer
SAS/SSS1 / X / X / X / X
TIC1 / X / X / X / X / X / X
BoQ1 / X / X / X
SET2
SSS1 / X / X / X / X
ODR4 / X / X / X
PSSA3 / X / X / X / X
PVAAS3 / X / X / X / X
Safe Schools3 / X / X / X / X
School Profile4 / X / X / X / X
Notes regarding PAPBS evaluators’ access to data. 1 = data completed via PBISAssessment website and automatically obtained by PAPBS evaluators via PBSEval.org secure website; 2 = will be administered minimally once every three to five years, or more frequently based on need for additional data and/or recognition purposes; 3 = data obtained by PA PBS evaluators via public access; 4 = school must supply to PAPBS evaluators
Data Measures
· SAS = Effective Behavior Support: Self-Assessment Survey (SAS; Sugai, Horner, & Todd, 2003, 2009)
· TIC = Team Implementation Checklist (TIC; Sugai, Horrner, Lewis-Palmer, Rossetto Kickey, 2001, 2011)
· BoQ = School-wide Benchmarks of Quality (BoQ; Kincaid, Childs, & George, 2010)
· SET = School-wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2005)
· SSS = School Safety Survey (SSS; Sprague, Colvin, & Irvin, 2002)
· ODR = Collected via SWIS (School-Wide Information System; Educational and Community Supports, 2010)
· PSSA = Pennsylvania School System of Assessment
· PVAAS = Pennsylvania Value Added Assessment System
· Safe Schools = Pennsylvania Safe Schools Online Database (https://www.safeschools.state.pa.us/)
· School Profile = Data specifically requested from schools including:
o Total number of students
o Student Attendance Rate
o Staff Attendance Rate
o LRE per 3 reporting categories
o Referrals for Special Education Eligibility Determination
o Newly Identified Special Education Identifications
o OSS / ISS
o Out-of-School Placements for Students
o Out-of-School Placements for Students with EB
o Collaborating MH agency
o ODRs – if not using SWIS; otherwise, these data provided via SWIS
o Number of Students with IEPs
1/7/2014 FINAL
Page 1